[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:50 dist-test-slave-77v9 krb5kdc[22320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:50 dist-test-slave-77v9 krb5kdc[22320](info): set up 2 sockets
Aug 06 19:41:50 dist-test-slave-77v9 krb5kdc[22320](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 19:41:51.518595 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.007s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:41:51 dist-test-slave-77v9 krb5kdc[22320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:41:51.730805 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:41921
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:41921
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:52.114193 22333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:52.114605 22333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:52.114701 22333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:52.122043 22333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:52.122153 22333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:52.122203 22333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:52.122264 22333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:52.122313 22333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:52.132601 22333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:41921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:41921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:52.134516 22333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:52.136770 22333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:52.156639 22338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:52.156769 22341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:52.156780 22339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:52.157588 22333 server_base.cc:1034] running on GCE node
I20260806 19:41:52.158763 22333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:52.161181 22333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:52.162492 22333 hybrid_clock.cc:648] HybridClock initialized: now 1786045312162388 us; error 99 us; skew 500 ppm
I20260806 19:41:52.163121 22333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:52 dist-test-slave-77v9 krb5kdc[22320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045312, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:52.171219 22333 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:41:52.173772 22333 webserver.cc:460] Webserver started at http://127.21.202.126:37853/ using document root <none> and password file <none>
I20260806 19:41:52.174818 22333 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:52.174933 22333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:52.175362 22333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:52.179896 22333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "b52f2fdf8fce4dd58cae3f7ac234099a"
format_stamp: "Formatted at 2026-08-06 19:41:52 on dist-test-slave-77v9"
server_key: "a4378ea60b5f1167bfa5bbb444ebc53e"
server_key_iv: "cad0688a1d69ae1c30007b9ec6671a8b"
server_key_version: "encryptionkey@0"
I20260806 19:41:52.180663 22333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "b52f2fdf8fce4dd58cae3f7ac234099a"
format_stamp: "Formatted at 2026-08-06 19:41:52 on dist-test-slave-77v9"
server_key: "a4378ea60b5f1167bfa5bbb444ebc53e"
server_key_iv: "cad0688a1d69ae1c30007b9ec6671a8b"
server_key_version: "encryptionkey@0"
I20260806 19:41:52.188795 22333 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260806 19:41:52.194787 22348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:52.197078 22333 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.003s
I20260806 19:41:52.197383 22333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "b52f2fdf8fce4dd58cae3f7ac234099a"
format_stamp: "Formatted at 2026-08-06 19:41:52 on dist-test-slave-77v9"
server_key: "a4378ea60b5f1167bfa5bbb444ebc53e"
server_key_iv: "cad0688a1d69ae1c30007b9ec6671a8b"
server_key_version: "encryptionkey@0"
I20260806 19:41:52.197577 22333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:52.210412 22333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:52.215821 22333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:52.216214 22333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:52.239696 22333 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:41921
I20260806 19:41:52.239732 22399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:41921 every 8 connection(s)
I20260806 19:41:52.242070 22333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:52.243398 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 22333
I20260806 19:41:52.243963 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:41:52.247969 22313 external_mini_cluster.cc:1428] Setting key 8e1da48c21753b4d958f919e6ec1ef14
I20260806 19:41:52.252113 22400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:52.272971 22400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a: Bootstrap starting.
I20260806 19:41:52.279126 22400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:52.281029 22400 log.cc:824] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a: Log is configured to *not* fsync() on all Append() calls
Aug 06 19:41:52 dist-test-slave-77v9 krb5kdc[22320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:41:52.285374 22400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a: No bootstrap required, opened a new log
I20260806 19:41:52.296902 22402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:52.256939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43298 (local address 127.21.202.126:41921)
0806 19:41:52.260906 (+ 3967us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:52.260945 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:52.261028 (+ 83us) server_negotiation.cc:407] Connection header received
0806 19:41:52.270763 (+ 9735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:52.270786 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:52.271495 (+ 709us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:52.272088 (+ 593us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:52.275709 (+ 3621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:52.277228 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:52.279016 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:52.279818 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:52.280103 (+ 285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:52.286062 (+ 5959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:52.286131 (+ 69us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:52.286198 (+ 67us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:52.286237 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:52.289928 (+ 3691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:52.290861 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:52.290869 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:52.290875 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:52.291097 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:52.291721 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:52.291727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:52.291730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:52.292514 (+ 784us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:52.293072 (+ 558us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:52.294095 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0806 19:41:52.294606 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3473,"thread_start_us":2951,"threads_started":1}
I20260806 19:41:52.297250 22400 raft_consensus.cc:348] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b52f2fdf8fce4dd58cae3f7ac234099a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41921 } }
I20260806 19:41:52.297914 22400 raft_consensus.cc:374] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:52.298157 22400 raft_consensus.cc:729] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b52f2fdf8fce4dd58cae3f7ac234099a, State: Initialized, Role: FOLLOWER
I20260806 19:41:52.299643 22400 consensus_queue.cc:260] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b52f2fdf8fce4dd58cae3f7ac234099a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41921 } }
I20260806 19:41:52.300097 22400 raft_consensus.cc:388] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:52.300422 22400 raft_consensus.cc:482] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:52.300668 22400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:52.303745 22400 raft_consensus.cc:504] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b52f2fdf8fce4dd58cae3f7ac234099a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41921 } }
I20260806 19:41:52.304733 22400 leader_election.cc:302] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b52f2fdf8fce4dd58cae3f7ac234099a; no voters:
I20260806 19:41:52.305615 22400 leader_election.cc:288] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:52.305898 22405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:52.306677 22405 raft_consensus.cc:686] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [term 1 LEADER]: Becoming Leader. State: Replica: b52f2fdf8fce4dd58cae3f7ac234099a, State: Running, Role: LEADER
I20260806 19:41:52.307413 22405 consensus_queue.cc:237] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b52f2fdf8fce4dd58cae3f7ac234099a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41921 } }
I20260806 19:41:52.308597 22400 sys_catalog.cc:564] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:52.313160 22406 sys_catalog.cc:455] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b52f2fdf8fce4dd58cae3f7ac234099a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b52f2fdf8fce4dd58cae3f7ac234099a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41921 } } }
I20260806 19:41:52.313566 22406 sys_catalog.cc:458] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:52.314822 22407 sys_catalog.cc:455] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b52f2fdf8fce4dd58cae3f7ac234099a. Latest consensus state: current_term: 1 leader_uuid: "b52f2fdf8fce4dd58cae3f7ac234099a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b52f2fdf8fce4dd58cae3f7ac234099a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41921 } } }
I20260806 19:41:52.315554 22407 sys_catalog.cc:458] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:52.317860 22413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:52.325922 22413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:52.339694 22413 catalog_manager.cc:1347] Generated new cluster ID: 36698a2b9a1140e3a7418781fb575a92
I20260806 19:41:52.339823 22413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:52.354585 22413 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:52.356853 22413 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:52.368764 22413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b52f2fdf8fce4dd58cae3f7ac234099a: Generated new TSK 0
I20260806 19:41:52.370205 22413 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:41:52.429759 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41921
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:52.777144 22426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:52.777426 22426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:52.777590 22426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:52.785172 22426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:52.785295 22426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:52.785472 22426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:41:52.795773 22426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41921
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:52.797240 22426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:52.798720 22426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:52.813351 22434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:52.813369 22431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:52.813464 22432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:52.814106 22426 server_base.cc:1034] running on GCE node
I20260806 19:41:52.814781 22426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:52.815953 22426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:52.817210 22426 hybrid_clock.cc:648] HybridClock initialized: now 1786045312817143 us; error 58 us; skew 500 ppm
I20260806 19:41:52.817572 22426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:52 dist-test-slave-77v9 krb5kdc[22320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045312, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:52.821113 22426 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:41:52.822757 22426 webserver.cc:460] Webserver started at http://127.21.202.65:37581/ using document root <none> and password file <none>
I20260806 19:41:52.823490 22426 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:52.823647 22426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:52.824054 22426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:52.827219 22426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "1553d9b8338844bcbc2162143cd7191e"
format_stamp: "Formatted at 2026-08-06 19:41:52 on dist-test-slave-77v9"
server_key: "b64bec3232e4bacdc4a0bc616c331326"
server_key_iv: "3ae17a24566daab269196e9bd6120db9"
server_key_version: "encryptionkey@0"
I20260806 19:41:52.828012 22426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "1553d9b8338844bcbc2162143cd7191e"
format_stamp: "Formatted at 2026-08-06 19:41:52 on dist-test-slave-77v9"
server_key: "b64bec3232e4bacdc4a0bc616c331326"
server_key_iv: "3ae17a24566daab269196e9bd6120db9"
server_key_version: "encryptionkey@0"
I20260806 19:41:52.833518 22426 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 19:41:52.837128 22442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:52.838604 22426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:41:52.838781 22426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "1553d9b8338844bcbc2162143cd7191e"
format_stamp: "Formatted at 2026-08-06 19:41:52 on dist-test-slave-77v9"
server_key: "b64bec3232e4bacdc4a0bc616c331326"
server_key_iv: "3ae17a24566daab269196e9bd6120db9"
server_key_version: "encryptionkey@0"
I20260806 19:41:52.839035 22426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:52.862478 22426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:52.867178 22426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:52.867496 22426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:52.868952 22426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:41:52.870697 22426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:41:52.870776 22426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:52.870932 22426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:41:52.870982 22426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:52.907277 22426 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:33171
I20260806 19:41:52.907385 22554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:33171 every 8 connection(s)
I20260806 19:41:52.908653 22426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:41:52.910859 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 22426
I20260806 19:41:52.911095 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:41:52.911816 22313 external_mini_cluster.cc:1428] Setting key 9c61c61818ce90e7ee8a964b4619390c
Aug 06 19:41:52 dist-test-slave-77v9 krb5kdc[22320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045312, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
I20260806 19:41:52.932009 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:52.912413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:41457 (local address 127.21.202.126:41921)
0806 19:41:52.912948 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:52.912954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:52.915508 (+ 2554us) server_negotiation.cc:407] Connection header received
0806 19:41:52.916106 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:52.916114 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:52.916223 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:52.916359 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:52.918117 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:52.919220 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:52.920803 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:52.921250 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:52.921406 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:52.925420 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:52.925425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:52.925428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:52.925462 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:52.928570 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:52.929457 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:52.929462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:52.929464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:52.929593 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:52.930048 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:52.930052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:52.930054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:52.930371 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:52.930528 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:52.931398 (+ 870us) server_negotiation.cc:299] Negotiation successful
0806 19:41:52.931634 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":270,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
I20260806 19:41:52.933264 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:52.913065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:41921 (local address 127.21.202.65:41457)
0806 19:41:52.913939 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:52.914003 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:52.915834 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:52.916601 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:52.916610 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:52.917339 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:52.917943 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:52.917960 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:52.919401 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:52.919405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:52.920644 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:52.920656 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:52.921501 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:52.921508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:52.921748 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:52.922518 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:52.922541 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:52.925175 (+ 2634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:52.928748 (+ 3573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:52.928758 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:52.928777 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:52.929311 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:52.929709 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:52.929714 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:52.929718 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:52.929917 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:52.930510 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:52.930519 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:52.930876 (+ 357us) client_negotiation.cc:769] Sending connection context
0806 19:41:52.931288 (+ 412us) client_negotiation.cc:240] Negotiation successful
0806 19:41:52.931850 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":324,"threads_started":1}
I20260806 19:41:52.939909 22555 heartbeater.cc:347] Connected to a master server at 127.21.202.126:41921
I20260806 19:41:52.940531 22555 heartbeater.cc:464] Registering TS with master...
I20260806 19:41:52.941551 22555 heartbeater.cc:511] Master 127.21.202.126:41921 requested a full tablet report, sending...
I20260806 19:41:52.945986 22365 ts_manager.cc:194] Registered new tserver with Master: 1553d9b8338844bcbc2162143cd7191e (127.21.202.65:33171)
I20260806 19:41:52.948246 22365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:41457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:41:52.961097 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41921
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:53.306136 22561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:53.306514 22561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:53.306630 22561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:53.314519 22561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:53.314667 22561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:53.314806 22561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:41:53.325259 22561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41921
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:53.326653 22561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:53.328179 22561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:53.342952 22566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:53.342921 22567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:53.343288 22569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:53.343987 22561 server_base.cc:1034] running on GCE node
I20260806 19:41:53.344578 22561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:53.345623 22561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:53.346838 22561 hybrid_clock.cc:648] HybridClock initialized: now 1786045313346781 us; error 39 us; skew 500 ppm
I20260806 19:41:53.347175 22561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:53 dist-test-slave-77v9 krb5kdc[22320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045313, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:53.350652 22561 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:41:53.352240 22561 webserver.cc:460] Webserver started at http://127.21.202.66:41897/ using document root <none> and password file <none>
I20260806 19:41:53.353066 22561 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:53.353200 22561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:53.353538 22561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:53.356554 22561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "ffe1b4ebad124583bd4c58d48954f79e"
format_stamp: "Formatted at 2026-08-06 19:41:53 on dist-test-slave-77v9"
server_key: "7c49e25dbceb76591bf6fa69b9cf1c04"
server_key_iv: "6e759a9a95c2c3980f7875343a37dff4"
server_key_version: "encryptionkey@0"
I20260806 19:41:53.357430 22561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "ffe1b4ebad124583bd4c58d48954f79e"
format_stamp: "Formatted at 2026-08-06 19:41:53 on dist-test-slave-77v9"
server_key: "7c49e25dbceb76591bf6fa69b9cf1c04"
server_key_iv: "6e759a9a95c2c3980f7875343a37dff4"
server_key_version: "encryptionkey@0"
I20260806 19:41:53.363101 22561 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:41:53.367142 22576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:53.368700 22561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:41:53.368952 22561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "ffe1b4ebad124583bd4c58d48954f79e"
format_stamp: "Formatted at 2026-08-06 19:41:53 on dist-test-slave-77v9"
server_key: "7c49e25dbceb76591bf6fa69b9cf1c04"
server_key_iv: "6e759a9a95c2c3980f7875343a37dff4"
server_key_version: "encryptionkey@0"
I20260806 19:41:53.369185 22561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:53.380875 22561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:53.385365 22561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:53.385669 22561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:53.386701 22561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:41:53.388178 22561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:41:53.388289 22561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:53.388447 22561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:41:53.388522 22561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:53.426335 22561 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:34525
I20260806 19:41:53.426417 22689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:34525 every 8 connection(s)
I20260806 19:41:53.427696 22561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:41:53.432338 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 22561
I20260806 19:41:53.432713 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:41:53.433462 22313 external_mini_cluster.cc:1428] Setting key 5663c87796c15c7331dcd04393e5362e
Aug 06 19:41:53 dist-test-slave-77v9 krb5kdc[22320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045313, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:41:53.452041 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:53.430885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:51775 (local address 127.21.202.126:41921)
0806 19:41:53.431196 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:53.431202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:53.432529 (+ 1327us) server_negotiation.cc:407] Connection header received
0806 19:41:53.434935 (+ 2406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:53.434941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:53.435015 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:53.435175 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:53.436744 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:53.437525 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:53.439265 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:53.439705 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:53.439842 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:53.443521 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:53.443537 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:53.443545 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:53.443630 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:53.448407 (+ 4777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:53.449473 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:53.449480 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:53.449485 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:53.449645 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:53.450124 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:53.450129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:53.450133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:53.450428 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:53.450605 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:53.451587 (+ 982us) server_negotiation.cc:299] Negotiation successful
0806 19:41:53.451779 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260806 19:41:53.453817 22692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:53.431397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:41921 (local address 127.21.202.66:51775)
0806 19:41:53.432342 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:53.432408 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:53.434672 (+ 2264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:53.435442 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:53.435452 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:53.435982 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:53.436573 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:53.436590 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:53.437678 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:53.437684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:53.439114 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:53.439131 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:53.439934 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:53.439942 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:53.440169 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:53.440928 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:53.440951 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:53.443223 (+ 2272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:53.448601 (+ 5378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:53.448613 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:53.448634 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:53.449321 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:53.449778 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:53.449783 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:53.449787 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:53.449996 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:53.450568 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:53.450579 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:53.451064 (+ 485us) client_negotiation.cc:769] Sending connection context
0806 19:41:53.451454 (+ 390us) client_negotiation.cc:240] Negotiation successful
0806 19:41:53.452070 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":353,"threads_started":1}
I20260806 19:41:53.455506 22690 heartbeater.cc:347] Connected to a master server at 127.21.202.126:41921
I20260806 19:41:53.455865 22690 heartbeater.cc:464] Registering TS with master...
I20260806 19:41:53.457059 22690 heartbeater.cc:511] Master 127.21.202.126:41921 requested a full tablet report, sending...
I20260806 19:41:53.458912 22365 ts_manager.cc:194] Registered new tserver with Master: ffe1b4ebad124583bd4c58d48954f79e (127.21.202.66:34525)
I20260806 19:41:53.459754 22365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:51775
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:41:53.493624 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41921
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:53.848541 22695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:53.848955 22695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:53.849192 22695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:53.857460 22695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:53.857571 22695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:53.857689 22695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:41:53.867718 22695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41921
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:53.869072 22695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:53.870602 22695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:53.884420 22700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:53.884479 22701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:53.884529 22703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:53.885351 22695 server_base.cc:1034] running on GCE node
I20260806 19:41:53.885972 22695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:53.886977 22695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:53.888280 22695 hybrid_clock.cc:648] HybridClock initialized: now 1786045313888210 us; error 56 us; skew 500 ppm
I20260806 19:41:53.888633 22695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:53 dist-test-slave-77v9 krb5kdc[22320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045313, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:53.891921 22695 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:41:53.893571 22695 webserver.cc:460] Webserver started at http://127.21.202.67:37855/ using document root <none> and password file <none>
I20260806 19:41:53.894255 22695 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:53.894387 22695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:53.894727 22695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:53.898025 22695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "906a5e77c6604de98a6fd1f1d6ebf5df"
format_stamp: "Formatted at 2026-08-06 19:41:53 on dist-test-slave-77v9"
server_key: "17cc7fa4363138ceddcf5bcef18a7b35"
server_key_iv: "cfe4d0ed7312c1e1fe6cf6a71695c474"
server_key_version: "encryptionkey@0"
I20260806 19:41:53.898918 22695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "906a5e77c6604de98a6fd1f1d6ebf5df"
format_stamp: "Formatted at 2026-08-06 19:41:53 on dist-test-slave-77v9"
server_key: "17cc7fa4363138ceddcf5bcef18a7b35"
server_key_iv: "cfe4d0ed7312c1e1fe6cf6a71695c474"
server_key_version: "encryptionkey@0"
I20260806 19:41:53.904594 22695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 19:41:53.908301 22710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:53.910004 22695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:41:53.910200 22695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "906a5e77c6604de98a6fd1f1d6ebf5df"
format_stamp: "Formatted at 2026-08-06 19:41:53 on dist-test-slave-77v9"
server_key: "17cc7fa4363138ceddcf5bcef18a7b35"
server_key_iv: "cfe4d0ed7312c1e1fe6cf6a71695c474"
server_key_version: "encryptionkey@0"
I20260806 19:41:53.910418 22695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:53.931885 22695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:53.936312 22695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:53.936627 22695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:53.937709 22695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:41:53.939078 22695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:41:53.939188 22695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:53.939354 22695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:41:53.939435 22695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:53.952636 22555 heartbeater.cc:503] Master 127.21.202.126:41921 was elected leader, sending a full tablet report...
I20260806 19:41:53.980285 22695 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:43349
I20260806 19:41:53.980329 22822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:43349 every 8 connection(s)
I20260806 19:41:53.981766 22695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:41:53.985306 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 22695
I20260806 19:41:53.985518 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:41:53.986028 22313 external_mini_cluster.cc:1428] Setting key 3de6558e1c1b12e4f7e571e4dba0511f
Aug 06 19:41:53 dist-test-slave-77v9 krb5kdc[22320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045313, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:41:54.004287 22825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:53.984450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:36029 (local address 127.21.202.126:41921)
0806 19:41:53.984998 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:53.985004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:53.986058 (+ 1054us) server_negotiation.cc:407] Connection header received
0806 19:41:53.988566 (+ 2508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:53.988575 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:53.988681 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:53.988793 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:53.991258 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:53.992080 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:53.993694 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:53.994105 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:53.994220 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:53.997861 (+ 3641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:53.997866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:53.997869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:53.997913 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:54.000504 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:54.001323 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:54.001328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:54.001330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:54.001569 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:54.002150 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:54.002156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:54.002158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:54.002534 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:54.002713 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:54.003575 (+ 862us) server_negotiation.cc:299] Negotiation successful
0806 19:41:54.003844 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":302,"threads_started":1}
I20260806 19:41:54.005369 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:53.984995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:41921 (local address 127.21.202.67:36029)
0806 19:41:53.985866 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:53.985915 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:53.988276 (+ 2361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:53.989173 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:53.989187 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:53.990122 (+ 935us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:53.991056 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:53.991077 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:53.992245 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:53.992250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:53.993546 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:53.993559 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:53.994347 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:53.994353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:53.994561 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:53.995271 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:53.995303 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:53.997638 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:54.000639 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:54.000646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:54.000658 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:54.001169 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:54.001727 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:54.001732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:54.001736 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:54.002006 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:54.002717 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:54.002727 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:54.003139 (+ 412us) client_negotiation.cc:769] Sending connection context
0806 19:41:54.003447 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 19:41:54.003986 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":367,"threads_started":1}
I20260806 19:41:54.006901 22823 heartbeater.cc:347] Connected to a master server at 127.21.202.126:41921
I20260806 19:41:54.007247 22823 heartbeater.cc:464] Registering TS with master...
I20260806 19:41:54.008000 22823 heartbeater.cc:511] Master 127.21.202.126:41921 requested a full tablet report, sending...
I20260806 19:41:54.009653 22365 ts_manager.cc:194] Registered new tserver with Master: 906a5e77c6604de98a6fd1f1d6ebf5df (127.21.202.67:43349)
I20260806 19:41:54.010577 22365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:36029
I20260806 19:41:54.016942 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 19:41:54 dist-test-slave-77v9 krb5kdc[22320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045314, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 19:41:54 dist-test-slave-77v9 krb5kdc[22320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045314, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:41:54.070749 22831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:54.055009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36688 (local address 127.21.202.65:33171)
0806 19:41:54.055815 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:54.055823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:54.055850 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:41:54.055965 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:54.055973 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:54.056191 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:54.056340 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:54.057933 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:54.058968 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:54.060192 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:54.060751 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:54.060937 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:54.064764 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:54.064775 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:54.064789 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:54.064879 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:54.067827 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:54.068507 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:54.068514 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:54.068517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:54.068644 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:54.069116 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:54.069124 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:54.069127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:54.069481 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:54.069792 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:54.070108 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 19:41:54.070384 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":510,"threads_started":1}
W20260806 19:41:54.072254 22470 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36688
Aug 06 19:41:54 dist-test-slave-77v9 krb5kdc[22320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 19:41:54 dist-test-slave-77v9 krb5kdc[22320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:41:54.121702 22831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:54.107211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36696 (local address 127.21.202.65:33171)
0806 19:41:54.107461 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:54.107469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:54.107657 (+ 188us) server_negotiation.cc:407] Connection header received
0806 19:41:54.107798 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:54.107802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:54.107914 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:54.108011 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:54.109526 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:54.110386 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:54.111626 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:54.112045 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:54.112142 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:54.116410 (+ 4268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:54.116416 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:54.116419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:54.116457 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:54.119131 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:54.119822 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:54.119826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:54.119829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:54.119963 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:54.120324 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:54.120327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:54.120329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:54.120630 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:54.120860 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:54.121205 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 19:41:54.121385 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 19:41:54.123801 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 22426
I20260806 19:41:54.137163 22831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:54.125055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36710 (local address 127.21.202.65:33171)
0806 19:41:54.125288 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:54.125294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:54.125512 (+ 218us) server_negotiation.cc:407] Connection header received
0806 19:41:54.125708 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:54.125713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:54.125784 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:54.125883 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:54.127094 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:54.127791 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:54.129133 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:54.129569 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:54.129709 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:54.131904 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:54.131909 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:54.131912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:54.131940 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:54.134675 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:54.135347 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:54.135353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:54.135356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:54.135514 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:54.135905 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:54.135913 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:54.135921 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:54.136247 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:54.136444 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:54.136681 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 19:41:54.136907 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 19:41:54.138024 22550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:54.227876 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 22426
I20260806 19:41:54.245870 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 22561
Aug 06 19:41:54 dist-test-slave-77v9 krb5kdc[22320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:41:54.261750 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:54.246886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39064 (local address 127.21.202.66:34525)
0806 19:41:54.247469 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:54.247476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:54.247502 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:41:54.247563 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:54.247569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:54.247767 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:54.247919 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:54.249133 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:54.250150 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:54.251363 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:54.251906 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:54.252065 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:54.256148 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:54.256159 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:54.256173 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:54.256208 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:54.259010 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:54.259683 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:54.259689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:54.259691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:54.259821 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:54.260160 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:54.260163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:54.260165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:54.260480 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:54.260709 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:54.261162 (+ 453us) server_negotiation.cc:299] Negotiation successful
0806 19:41:54.261356 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":306,"threads_started":1}
I20260806 19:41:54.263131 22685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:54.345844 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 22561
I20260806 19:41:54.363586 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 22695
Aug 06 19:41:54 dist-test-slave-77v9 krb5kdc[22320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:41:54.381937 22840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:54.364671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56350 (local address 127.21.202.67:43349)
0806 19:41:54.365383 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:54.365390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:54.365422 (+ 32us) server_negotiation.cc:407] Connection header received
0806 19:41:54.365489 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:54.365496 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:54.365709 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:54.365901 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:54.367301 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:54.368340 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:54.369726 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:54.370312 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:54.370472 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:54.374870 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:54.374882 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:54.374897 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:54.374936 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:54.378496 (+ 3560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:54.379312 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:54.379322 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:54.379324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:54.379471 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:54.379939 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:54.379944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:54.379946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:54.380340 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:54.380687 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:54.381225 (+ 538us) server_negotiation.cc:299] Negotiation successful
0806 19:41:54.381530 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":376,"threads_started":1}
I20260806 19:41:54.383314 22818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:54.470604 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 22695
I20260806 19:41:54.487979 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 22333
I20260806 19:41:54.489776 22395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:54.594861 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 22333
2026-08-06T19:41:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4217 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:54 dist-test-slave-77v9 krb5kdc[22845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:54 dist-test-slave-77v9 krb5kdc[22845](info): set up 2 sockets
Aug 06 19:41:54 dist-test-slave-77v9 krb5kdc[22845](info): commencing operation
krb5kdc: starting...
W20260806 19:41:55.726198 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:41:55 dist-test-slave-77v9 krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:41:55.941191 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:34671
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:34671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:56.275272 22858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:56.275581 22858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:56.275672 22858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:56.282984 22858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:56.283106 22858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:56.283154 22858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:56.283208 22858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:56.283253 22858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:56.293429 22858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:34671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:34671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:56.294780 22858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:56.296532 22858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:56.309355 22863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:56.309440 22864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:56.309270 22866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:56.310714 22858 server_base.cc:1034] running on GCE node
I20260806 19:41:56.311554 22858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:56.313148 22858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:56.314419 22858 hybrid_clock.cc:648] HybridClock initialized: now 1786045316314346 us; error 61 us; skew 500 ppm
I20260806 19:41:56.314854 22858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:56 dist-test-slave-77v9 krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045316, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:56.318430 22858 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:41:56.320210 22858 webserver.cc:460] Webserver started at http://127.21.202.126:36027/ using document root <none> and password file <none>
I20260806 19:41:56.320999 22858 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:56.321151 22858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:56.321489 22858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:56.324555 22858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "f728e316a38743969a779b2b0fd59753"
format_stamp: "Formatted at 2026-08-06 19:41:56 on dist-test-slave-77v9"
server_key: "6056dd9f77889ab88cc6d1b6d1a85beb"
server_key_iv: "5b28e7cdae604d88241f775a5156b634"
server_key_version: "encryptionkey@0"
I20260806 19:41:56.325426 22858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "f728e316a38743969a779b2b0fd59753"
format_stamp: "Formatted at 2026-08-06 19:41:56 on dist-test-slave-77v9"
server_key: "6056dd9f77889ab88cc6d1b6d1a85beb"
server_key_iv: "5b28e7cdae604d88241f775a5156b634"
server_key_version: "encryptionkey@0"
I20260806 19:41:56.331162 22858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:41:56.334733 22873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:56.336210 22858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 19:41:56.336426 22858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "f728e316a38743969a779b2b0fd59753"
format_stamp: "Formatted at 2026-08-06 19:41:56 on dist-test-slave-77v9"
server_key: "6056dd9f77889ab88cc6d1b6d1a85beb"
server_key_iv: "5b28e7cdae604d88241f775a5156b634"
server_key_version: "encryptionkey@0"
I20260806 19:41:56.336669 22858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:56.352911 22858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:56.357437 22858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:56.357753 22858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:56.379235 22858 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:34671
I20260806 19:41:56.379253 22924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:34671 every 8 connection(s)
I20260806 19:41:56.380741 22858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:56.381453 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 22858
I20260806 19:41:56.381721 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:41:56.382570 22313 external_mini_cluster.cc:1428] Setting key 4a7cf7b55da2b092a6ecfb9cfb8271c1
I20260806 19:41:56.385437 22925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 19:41:56 dist-test-slave-77v9 krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:41:56.399201 22925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753: Bootstrap starting.
I20260806 19:41:56.403105 22925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:56.404217 22925 log.cc:824] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:56.407691 22925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753: No bootstrap required, opened a new log
I20260806 19:41:56.407953 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:56.384793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56244 (local address 127.21.202.126:34671)
0806 19:41:56.385691 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:56.385703 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:56.385747 (+ 44us) server_negotiation.cc:407] Connection header received
0806 19:41:56.387372 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:56.387394 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:56.387795 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:56.388148 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:56.390106 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:56.391526 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:56.393341 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:56.394074 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:56.394207 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:56.399263 (+ 5056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:56.399272 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:56.399289 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:56.399332 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:56.403138 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:56.403994 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:56.404007 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:56.404014 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:56.404225 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:56.404645 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:56.404650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:56.404652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:56.405528 (+ 876us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:56.405842 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:56.406437 (+ 595us) server_negotiation.cc:299] Negotiation successful
0806 19:41:56.406890 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":374,"threads_started":1}
I20260806 19:41:56.416290 22925 raft_consensus.cc:348] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f728e316a38743969a779b2b0fd59753" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34671 } }
I20260806 19:41:56.416790 22925 raft_consensus.cc:374] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:56.416939 22925 raft_consensus.cc:729] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f728e316a38743969a779b2b0fd59753, State: Initialized, Role: FOLLOWER
I20260806 19:41:56.417644 22925 consensus_queue.cc:260] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f728e316a38743969a779b2b0fd59753" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34671 } }
I20260806 19:41:56.417938 22925 raft_consensus.cc:388] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:56.418073 22925 raft_consensus.cc:482] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:56.418267 22925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:56.420809 22925 raft_consensus.cc:504] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f728e316a38743969a779b2b0fd59753" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34671 } }
I20260806 19:41:56.421540 22925 leader_election.cc:302] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f728e316a38743969a779b2b0fd59753; no voters:
I20260806 19:41:56.422348 22925 leader_election.cc:288] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:56.422470 22930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:56.423207 22930 raft_consensus.cc:686] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [term 1 LEADER]: Becoming Leader. State: Replica: f728e316a38743969a779b2b0fd59753, State: Running, Role: LEADER
I20260806 19:41:56.423812 22930 consensus_queue.cc:237] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f728e316a38743969a779b2b0fd59753" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34671 } }
I20260806 19:41:56.425029 22925 sys_catalog.cc:564] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:56.428136 22932 sys_catalog.cc:455] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f728e316a38743969a779b2b0fd59753. Latest consensus state: current_term: 1 leader_uuid: "f728e316a38743969a779b2b0fd59753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f728e316a38743969a779b2b0fd59753" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34671 } } }
I20260806 19:41:56.428479 22931 sys_catalog.cc:455] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f728e316a38743969a779b2b0fd59753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f728e316a38743969a779b2b0fd59753" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34671 } } }
I20260806 19:41:56.428670 22932 sys_catalog.cc:458] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:56.428752 22931 sys_catalog.cc:458] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:56.429733 22938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:56.434366 22938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:56.445909 22938 catalog_manager.cc:1347] Generated new cluster ID: edf3b2698aa24ffda6e2b499af76fe75
I20260806 19:41:56.446050 22938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:56.461257 22938 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:56.463568 22938 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:56.481678 22938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f728e316a38743969a779b2b0fd59753: Generated new TSK 0
I20260806 19:41:56.482657 22938 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:41:56.537896 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34671
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:56.896715 22951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:56.897111 22951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:56.897310 22951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:56.905011 22951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:56.905129 22951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:56.905349 22951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:41:56.916034 22951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34671
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:56.917517 22951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:56.918960 22951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:56.932792 22956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:56.932835 22957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:56.933485 22959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:56.934242 22951 server_base.cc:1034] running on GCE node
I20260806 19:41:56.934893 22951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:56.935913 22951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:56.937150 22951 hybrid_clock.cc:648] HybridClock initialized: now 1786045316937106 us; error 36 us; skew 500 ppm
I20260806 19:41:56.937551 22951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:56 dist-test-slave-77v9 krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045316, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:56.940932 22951 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:41:56.946233 22951 webserver.cc:460] Webserver started at http://127.21.202.65:43203/ using document root <none> and password file <none>
I20260806 19:41:56.947109 22951 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:56.947288 22951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:56.947650 22951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:56.950812 22951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "84773e3a70164836b1ded49427541bba"
format_stamp: "Formatted at 2026-08-06 19:41:56 on dist-test-slave-77v9"
server_key: "955dada4d0f907465747a891edace3e0"
server_key_iv: "71112a5f0dd2aa1e27baa5699921bad5"
server_key_version: "encryptionkey@0"
I20260806 19:41:56.951630 22951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "84773e3a70164836b1ded49427541bba"
format_stamp: "Formatted at 2026-08-06 19:41:56 on dist-test-slave-77v9"
server_key: "955dada4d0f907465747a891edace3e0"
server_key_iv: "71112a5f0dd2aa1e27baa5699921bad5"
server_key_version: "encryptionkey@0"
I20260806 19:41:56.957082 22951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 19:41:56.960767 22966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:56.962373 22951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:41:56.962558 22951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "84773e3a70164836b1ded49427541bba"
format_stamp: "Formatted at 2026-08-06 19:41:56 on dist-test-slave-77v9"
server_key: "955dada4d0f907465747a891edace3e0"
server_key_iv: "71112a5f0dd2aa1e27baa5699921bad5"
server_key_version: "encryptionkey@0"
I20260806 19:41:56.962816 22951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:56.979784 22951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:56.984263 22951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:56.984560 22951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:56.985630 22951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:41:56.987061 22951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:41:56.987165 22951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:56.987316 22951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:41:56.987393 22951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:57.024731 22951 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:45063
I20260806 19:41:57.024781 23078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:45063 every 8 connection(s)
I20260806 19:41:57.026517 22951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:41:57.032210 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 22951
I20260806 19:41:57.032476 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:41:57.033310 22313 external_mini_cluster.cc:1428] Setting key bf77878efad32d6c7d6d82bbc786c9ca
Aug 06 19:41:57 dist-test-slave-77v9 krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045316, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:41:57.053610 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:57.029399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:47007 (local address 127.21.202.126:34671)
0806 19:41:57.029908 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:57.029913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:57.031387 (+ 1474us) server_negotiation.cc:407] Connection header received
0806 19:41:57.034152 (+ 2765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:57.034158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:57.034281 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:57.034412 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:57.036595 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:57.037861 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:57.040126 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:57.040880 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:57.041043 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:57.045930 (+ 4887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:57.045939 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:57.045947 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:57.045996 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:57.049902 (+ 3906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:57.050717 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:57.050726 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:57.050732 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:57.050895 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:57.051407 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:57.051415 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:57.051421 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:57.051824 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:57.051990 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:57.052755 (+ 765us) server_negotiation.cc:299] Negotiation successful
0806 19:41:57.053063 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":279,"threads_started":1}
I20260806 19:41:57.054364 23082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:57.029918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34671 (local address 127.21.202.65:47007)
0806 19:41:57.031200 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:57.031253 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:57.033821 (+ 2568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:57.034766 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:57.034778 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:57.035645 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:57.036301 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:57.036319 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:57.038092 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:57.038097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:57.039967 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:57.039986 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:57.041399 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:57.041409 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:57.041752 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:57.042600 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:57.042635 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:57.045667 (+ 3032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:57.050091 (+ 4424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:57.050099 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:57.050114 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:57.050576 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:57.051052 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:57.051056 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:57.051059 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:57.051268 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:57.052001 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:57.052008 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:57.052356 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 19:41:57.052597 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 19:41:57.053135 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"thread_start_us":615,"threads_started":1}
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
I20260806 19:41:57.056021 23079 heartbeater.cc:347] Connected to a master server at 127.21.202.126:34671
I20260806 19:41:57.056366 23079 heartbeater.cc:464] Registering TS with master...
I20260806 19:41:57.057232 23079 heartbeater.cc:511] Master 127.21.202.126:34671 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
I20260806 19:41:57.059598 22890 ts_manager.cc:194] Registered new tserver with Master: 84773e3a70164836b1ded49427541bba (127.21.202.65:45063)
I20260806 19:41:57.061892 22890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:47007
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:41:57.085171 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34671
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:57.424793 23085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:57.425109 23085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:57.425280 23085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:57.436427 23085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:57.436545 23085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:57.436664 23085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:41:57.447136 23085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34671
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:57.448488 23085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:57.449909 23085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:57.464339 23090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:57.464368 23093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:57.464478 23091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:57.466203 23085 server_base.cc:1034] running on GCE node
I20260806 19:41:57.467180 23085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:57.468221 23085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:57.469514 23085 hybrid_clock.cc:648] HybridClock initialized: now 1786045317469407 us; error 42 us; skew 500 ppm
I20260806 19:41:57.469910 23085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:57 dist-test-slave-77v9 krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045317, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:57.477255 23085 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:41:57.479055 23085 webserver.cc:460] Webserver started at http://127.21.202.66:41029/ using document root <none> and password file <none>
I20260806 19:41:57.479832 23085 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:57.479997 23085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:57.480361 23085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:57.483788 23085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "789f1fb0206b46cd81432fbb763eedde"
format_stamp: "Formatted at 2026-08-06 19:41:57 on dist-test-slave-77v9"
server_key: "65122bdea5d55d90d5cacab64fa2212a"
server_key_iv: "1fda55ffc7f23237d22605d77ed20a2b"
server_key_version: "encryptionkey@0"
I20260806 19:41:57.484663 23085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "789f1fb0206b46cd81432fbb763eedde"
format_stamp: "Formatted at 2026-08-06 19:41:57 on dist-test-slave-77v9"
server_key: "65122bdea5d55d90d5cacab64fa2212a"
server_key_iv: "1fda55ffc7f23237d22605d77ed20a2b"
server_key_version: "encryptionkey@0"
I20260806 19:41:57.490819 23085 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260806 19:41:57.494611 23100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:57.496084 23085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 19:41:57.496287 23085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "789f1fb0206b46cd81432fbb763eedde"
format_stamp: "Formatted at 2026-08-06 19:41:57 on dist-test-slave-77v9"
server_key: "65122bdea5d55d90d5cacab64fa2212a"
server_key_iv: "1fda55ffc7f23237d22605d77ed20a2b"
server_key_version: "encryptionkey@0"
I20260806 19:41:57.496665 23085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:57.518932 23085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:57.523748 23085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:57.524109 23085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:57.525321 23085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:41:57.527042 23085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:41:57.527161 23085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:57.527326 23085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:41:57.527415 23085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:57.564927 23085 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:38213
I20260806 19:41:57.565023 23212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:38213 every 8 connection(s)
I20260806 19:41:57.566332 23085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:41:57.567233 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 23085
I20260806 19:41:57.567477 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:41:57.568049 22313 external_mini_cluster.cc:1428] Setting key 4f3801f48fff77baffe0e09c65880b00
Aug 06 19:41:57 dist-test-slave-77v9 krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045317, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
I20260806 19:41:57.590263 23215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:57.569382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:54221 (local address 127.21.202.126:34671)
0806 19:41:57.569971 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:57.569977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:57.571363 (+ 1386us) server_negotiation.cc:407] Connection header received
0806 19:41:57.573774 (+ 2411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:57.573782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:57.573934 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:57.574060 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:57.576524 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:57.577459 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:57.579184 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:57.579678 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:57.579826 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:57.583804 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:57.583810 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:57.583813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:57.583849 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:57.586623 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:57.587486 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:57.587492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:57.587497 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:57.587631 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:57.588083 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:57.588090 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:57.588093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:57.588388 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:57.588542 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:57.589454 (+ 912us) server_negotiation.cc:299] Negotiation successful
0806 19:41:57.589691 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":271,"threads_started":1}
I20260806 19:41:57.591370 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:57.570226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34671 (local address 127.21.202.66:54221)
0806 19:41:57.571154 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:57.571209 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:57.573448 (+ 2239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:57.574395 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:57.574407 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:57.575348 (+ 941us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:57.576321 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:57.576344 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:57.577637 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:57.577643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:57.579021 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:57.579034 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:57.579968 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:57.579976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:57.580239 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:57.581062 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:57.581092 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:57.583527 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:57.586791 (+ 3264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:57.586801 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:57.586821 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:57.587346 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:57.587748 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:57.587754 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:57.587757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:57.587965 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:57.588528 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:57.588536 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:57.589024 (+ 488us) client_negotiation.cc:769] Sending connection context
0806 19:41:57.589374 (+ 350us) client_negotiation.cc:240] Negotiation successful
0806 19:41:57.589948 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":385,"threads_started":1}
I20260806 19:41:57.593179 23213 heartbeater.cc:347] Connected to a master server at 127.21.202.126:34671
I20260806 19:41:57.593569 23213 heartbeater.cc:464] Registering TS with master...
I20260806 19:41:57.594399 23213 heartbeater.cc:511] Master 127.21.202.126:34671 requested a full tablet report, sending...
I20260806 19:41:57.596071 22890 ts_manager.cc:194] Registered new tserver with Master: 789f1fb0206b46cd81432fbb763eedde (127.21.202.66:38213)
I20260806 19:41:57.597090 22890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:54221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:41:57.617480 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34671
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:57.958956 23219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:57.959254 23219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:57.959420 23219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:57.967227 23219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:57.967372 23219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:57.967628 23219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:41:57.978400 23219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34671
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:57.979849 23219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:57.981346 23219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:57.995683 23224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:57.995774 23225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:57.995827 23227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:57.997012 23219 server_base.cc:1034] running on GCE node
I20260806 19:41:57.997795 23219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:57.998854 23219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:58.000126 23219 hybrid_clock.cc:648] HybridClock initialized: now 1786045318000060 us; error 53 us; skew 500 ppm
I20260806 19:41:58.000558 23219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:57 dist-test-slave-77v9 krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045317, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:58.003994 23219 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:41:58.005600 23219 webserver.cc:460] Webserver started at http://127.21.202.67:34007/ using document root <none> and password file <none>
I20260806 19:41:58.006305 23219 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:58.006471 23219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:58.006793 23219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:58.010028 23219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "f8f31eae9c854632ae62de37142b1b18"
format_stamp: "Formatted at 2026-08-06 19:41:58 on dist-test-slave-77v9"
server_key: "3d741ea5e7e2f8d0c0a32b66d8e2d727"
server_key_iv: "2da1cb866465cd445e2c5b4caf703233"
server_key_version: "encryptionkey@0"
I20260806 19:41:58.010846 23219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "f8f31eae9c854632ae62de37142b1b18"
format_stamp: "Formatted at 2026-08-06 19:41:58 on dist-test-slave-77v9"
server_key: "3d741ea5e7e2f8d0c0a32b66d8e2d727"
server_key_iv: "2da1cb866465cd445e2c5b4caf703233"
server_key_version: "encryptionkey@0"
I20260806 19:41:58.016353 23219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 19:41:58.020231 23234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:58.021756 23219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:41:58.021957 23219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "f8f31eae9c854632ae62de37142b1b18"
format_stamp: "Formatted at 2026-08-06 19:41:58 on dist-test-slave-77v9"
server_key: "3d741ea5e7e2f8d0c0a32b66d8e2d727"
server_key_iv: "2da1cb866465cd445e2c5b4caf703233"
server_key_version: "encryptionkey@0"
I20260806 19:41:58.022188 23219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:58.057817 23219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:58.062851 23219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:58.063194 23219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:58.064324 23219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:41:58.065124 23079 heartbeater.cc:503] Master 127.21.202.126:34671 was elected leader, sending a full tablet report...
I20260806 19:41:58.066051 23219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:41:58.066159 23219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:58.066298 23219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:41:58.066380 23219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:58.103996 23219 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:44779
I20260806 19:41:58.104053 23346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:44779 every 8 connection(s)
I20260806 19:41:58.105530 23219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:41:58.109622 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 23219
I20260806 19:41:58.109995 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:41:58.110558 22313 external_mini_cluster.cc:1428] Setting key 175e348fcdc8d2faea89014cf2c8fd0d
Aug 06 19:41:58 dist-test-slave-77v9 krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045317, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:41:58.127533 23349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.108713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:51333 (local address 127.21.202.126:34671)
0806 19:41:58.109203 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:58.109231 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:58.110357 (+ 1126us) server_negotiation.cc:407] Connection header received
0806 19:41:58.112246 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:58.112251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:58.112352 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:58.112467 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:58.114419 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.115214 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.116760 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.117290 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.117409 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.121173 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:58.121178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:58.121181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:58.121225 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:58.124102 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.124939 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.124943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.124945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.125070 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.125632 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.125639 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.125644 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.125954 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:58.126107 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:58.126999 (+ 892us) server_negotiation.cc:299] Negotiation successful
0806 19:41:58.127196 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":241,"threads_started":1}
I20260806 19:41:58.128686 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.109358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34671 (local address 127.21.202.67:51333)
0806 19:41:58.110164 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:58.110212 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:58.111981 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:58.112779 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:58.112790 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:58.113618 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:58.114258 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:58.114276 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.115358 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.115361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:58.116583 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:58.116594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.117535 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.117540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:58.117740 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.118495 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:58.118519 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:58.120916 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:58.124260 (+ 3344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:58.124267 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:58.124287 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:58.124697 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:58.125217 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:58.125222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:58.125225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:58.125493 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:58.126088 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:58.126098 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:58.126562 (+ 464us) client_negotiation.cc:769] Sending connection context
0806 19:41:58.126865 (+ 303us) client_negotiation.cc:240] Negotiation successful
0806 19:41:58.127323 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":353,"threads_started":1}
I20260806 19:41:58.130220 23347 heartbeater.cc:347] Connected to a master server at 127.21.202.126:34671
I20260806 19:41:58.130553 23347 heartbeater.cc:464] Registering TS with master...
I20260806 19:41:58.131318 23347 heartbeater.cc:511] Master 127.21.202.126:34671 requested a full tablet report, sending...
I20260806 19:41:58.132788 22890 ts_manager.cc:194] Registered new tserver with Master: f8f31eae9c854632ae62de37142b1b18 (127.21.202.67:44779)
I20260806 19:41:58.133659 22890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:51333
I20260806 19:41:58.139313 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:41:58.160606 23349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.146961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56250 (local address 127.21.202.126:34671)
0806 19:41:58.147137 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:58.147142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:58.147671 (+ 529us) server_negotiation.cc:407] Connection header received
0806 19:41:58.147880 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:58.147886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:58.147952 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:58.148061 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:58.149646 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.150402 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.151687 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.152114 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.152253 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.154763 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:58.154773 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:58.154776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:58.154808 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:58.157643 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.158376 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.158382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.158386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.158535 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.158961 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.158969 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.158973 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.159328 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:58.159616 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:58.159994 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 19:41:58.160256 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 19:41:58.170080 22890 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:41:58.179318 22890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:41:58.206537 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.196853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33078 (local address 127.21.202.65:45063)
0806 19:41:58.197570 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:58.197579 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:58.197609 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:41:58.197980 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:58.197996 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:58.198270 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:58.198477 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:41:58.199312 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.200915 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.204028 (+ 3113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.205251 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.205421 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.205675 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:58.205960 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 19:41:58.206134 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":326,"threads_started":1}
I20260806 19:41:58.206667 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.196608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:45063 (local address 127.0.0.1:33078)
0806 19:41:58.197396 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:58.197443 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:58.197671 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:58.198753 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:58.198759 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:58.198782 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:41:58.199103 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:58.199117 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.201067 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.201075 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:58.203845 (+ 2770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:58.203862 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.205461 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.205467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:58.205680 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.205759 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 19:41:58.205918 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 19:41:58.206184 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":56,"thread_start_us":343,"threads_started":1}
I20260806 19:41:58.209867 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.198108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54724 (local address 127.21.202.66:38213)
0806 19:41:58.201507 (+ 3399us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:58.201516 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:58.201543 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:41:58.201641 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:58.201648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:58.201894 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:58.202098 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:41:58.202884 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.204382 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.207506 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.208773 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.208962 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.209157 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:58.209414 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 19:41:58.209565 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3238,"thread_start_us":3137,"threads_started":1}
I20260806 19:41:58.209928 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.197783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:38213 (local address 127.0.0.1:54724)
0806 19:41:58.198377 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:58.198397 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:58.198511 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:58.202107 (+ 3596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:58.202112 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:58.202158 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:41:58.202651 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:58.202664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.204524 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.204530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:58.207311 (+ 2781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:58.207327 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.209038 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.209045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:58.209222 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.209250 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 19:41:58.209358 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 19:41:58.209492 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":22,"spinlock_wait_cycles":7680,"thread_start_us":2619,"threads_started":1}
I20260806 19:41:58.211380 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.198020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:44779 (local address 127.0.0.1:59516)
0806 19:41:58.201990 (+ 3970us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:58.202005 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:58.202161 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:58.203119 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:58.203124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:58.203150 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:41:58.203593 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:58.203604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.205470 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.205482 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:58.208054 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:58.208068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.210555 (+ 2487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.210561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:58.210725 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.210758 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 19:41:58.210869 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 19:41:58.210994 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3866,"mutex_wait_us":25,"spinlock_wait_cycles":7296,"thread_start_us":3627,"threads_started":1}
I20260806 19:41:58.212216 23014 tablet_service.cc:1467] Processing CreateTablet for tablet dd4723400f8b4ef99b4246a665f482dd (DEFAULT_TABLE table=test-table [id=5ee0fdce65274c3b942d72aad2067f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:41:58.213218 23148 tablet_service.cc:1467] Processing CreateTablet for tablet dd4723400f8b4ef99b4246a665f482dd (DEFAULT_TABLE table=test-table [id=5ee0fdce65274c3b942d72aad2067f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:41:58.213934 23014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd4723400f8b4ef99b4246a665f482dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:58.214351 23148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd4723400f8b4ef99b4246a665f482dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:58.217461 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.198903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59516 (local address 127.21.202.67:44779)
0806 19:41:58.200515 (+ 1612us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:58.200523 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:58.202318 (+ 1795us) server_negotiation.cc:407] Connection header received
0806 19:41:58.202424 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:58.202433 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:58.202667 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:58.202836 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:41:58.203782 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.205250 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.208219 (+ 2969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.209940 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.210211 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.210424 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:58.216878 (+ 6454us) server_negotiation.cc:299] Negotiation successful
0806 19:41:58.217088 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1444,"thread_start_us":1261,"threads_started":1}
I20260806 19:41:58.221295 23282 tablet_service.cc:1467] Processing CreateTablet for tablet dd4723400f8b4ef99b4246a665f482dd (DEFAULT_TABLE table=test-table [id=5ee0fdce65274c3b942d72aad2067f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:41:58.222785 23282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd4723400f8b4ef99b4246a665f482dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:58.227578 23366 tablet_bootstrap.cc:492] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde: Bootstrap starting.
I20260806 19:41:58.230763 23366 tablet_bootstrap.cc:654] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:58.231585 23367 tablet_bootstrap.cc:492] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba: Bootstrap starting.
I20260806 19:41:58.232103 23366 log.cc:824] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:58.235085 23367 tablet_bootstrap.cc:654] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:58.235496 23366 tablet_bootstrap.cc:492] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde: No bootstrap required, opened a new log
I20260806 19:41:58.235792 23366 ts_tablet_manager.cc:1397] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 19:41:58.236428 23367 log.cc:824] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:58.239742 23367 tablet_bootstrap.cc:492] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba: No bootstrap required, opened a new log
I20260806 19:41:58.240021 23367 ts_tablet_manager.cc:1397] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260806 19:41:58.244360 23370 tablet_bootstrap.cc:492] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18: Bootstrap starting.
I20260806 19:41:58.246605 23366 raft_consensus.cc:348] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84773e3a70164836b1ded49427541bba" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 45063 } } peers { permanent_uuid: "789f1fb0206b46cd81432fbb763eedde" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38213 } } peers { permanent_uuid: "f8f31eae9c854632ae62de37142b1b18" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44779 } }
I20260806 19:41:58.247265 23366 raft_consensus.cc:374] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:58.247401 23366 raft_consensus.cc:729] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 789f1fb0206b46cd81432fbb763eedde, State: Initialized, Role: FOLLOWER
I20260806 19:41:58.248104 23370 tablet_bootstrap.cc:654] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:58.248159 23366 consensus_queue.cc:260] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84773e3a70164836b1ded49427541bba" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 45063 } } peers { permanent_uuid: "789f1fb0206b46cd81432fbb763eedde" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38213 } } peers { permanent_uuid: "f8f31eae9c854632ae62de37142b1b18" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44779 } }
I20260806 19:41:58.249598 23213 heartbeater.cc:503] Master 127.21.202.126:34671 was elected leader, sending a full tablet report...
I20260806 19:41:58.249576 23370 log.cc:824] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:58.250205 23366 ts_tablet_manager.cc:1428] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde: Time spent starting tablet: real 0.014s user 0.005s sys 0.011s
I20260806 19:41:58.251412 23367 raft_consensus.cc:348] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84773e3a70164836b1ded49427541bba" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 45063 } } peers { permanent_uuid: "789f1fb0206b46cd81432fbb763eedde" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38213 } } peers { permanent_uuid: "f8f31eae9c854632ae62de37142b1b18" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44779 } }
I20260806 19:41:58.251920 23367 raft_consensus.cc:374] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:58.252053 23367 raft_consensus.cc:729] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84773e3a70164836b1ded49427541bba, State: Initialized, Role: FOLLOWER
I20260806 19:41:58.252745 23367 consensus_queue.cc:260] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84773e3a70164836b1ded49427541bba" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 45063 } } peers { permanent_uuid: "789f1fb0206b46cd81432fbb763eedde" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38213 } } peers { permanent_uuid: "f8f31eae9c854632ae62de37142b1b18" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44779 } }
I20260806 19:41:58.254278 23370 tablet_bootstrap.cc:492] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18: No bootstrap required, opened a new log
I20260806 19:41:58.254575 23370 ts_tablet_manager.cc:1397] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260806 19:41:58.254937 23367 ts_tablet_manager.cc:1428] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260806 19:41:58.263108 23370 raft_consensus.cc:348] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84773e3a70164836b1ded49427541bba" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 45063 } } peers { permanent_uuid: "789f1fb0206b46cd81432fbb763eedde" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38213 } } peers { permanent_uuid: "f8f31eae9c854632ae62de37142b1b18" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44779 } }
I20260806 19:41:58.263468 23370 raft_consensus.cc:374] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:58.263603 23370 raft_consensus.cc:729] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8f31eae9c854632ae62de37142b1b18, State: Initialized, Role: FOLLOWER
I20260806 19:41:58.264246 23370 consensus_queue.cc:260] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84773e3a70164836b1ded49427541bba" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 45063 } } peers { permanent_uuid: "789f1fb0206b46cd81432fbb763eedde" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38213 } } peers { permanent_uuid: "f8f31eae9c854632ae62de37142b1b18" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44779 } }
I20260806 19:41:58.265462 23347 heartbeater.cc:503] Master 127.21.202.126:34671 was elected leader, sending a full tablet report...
I20260806 19:41:58.266145 23370 ts_tablet_manager.cc:1428] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18: Time spent starting tablet: real 0.011s user 0.003s sys 0.009s
W20260806 19:41:58.279080 23080 tablet.cc:2369] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:41:58.318746 23214 tablet.cc:2369] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:41:58.336489 23373 raft_consensus.cc:482] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:41:58.336899 23373 raft_consensus.cc:504] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84773e3a70164836b1ded49427541bba" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 45063 } } peers { permanent_uuid: "789f1fb0206b46cd81432fbb763eedde" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38213 } } peers { permanent_uuid: "f8f31eae9c854632ae62de37142b1b18" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44779 } }
I20260806 19:41:58.339447 23373 leader_election.cc:288] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 789f1fb0206b46cd81432fbb763eedde (127.21.202.66:38213), f8f31eae9c854632ae62de37142b1b18 (127.21.202.67:44779)
I20260806 19:41:58.347360 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.340047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:57103 (local address 127.21.202.67:44779)
0806 19:41:58.340305 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:58.340315 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:58.341441 (+ 1126us) server_negotiation.cc:407] Connection header received
0806 19:41:58.341691 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:58.341707 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:58.341783 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:58.341898 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:41:58.342595 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.343395 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.345288 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.346168 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.346310 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.346368 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:58.346932 (+ 564us) server_negotiation.cc:299] Negotiation successful
0806 19:41:58.347088 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 19:41:58.347491 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.340018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:51535 (local address 127.21.202.66:38213)
0806 19:41:58.340250 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:58.340259 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:58.341355 (+ 1096us) server_negotiation.cc:407] Connection header received
0806 19:41:58.341719 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:58.341724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:58.341788 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:58.341890 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:41:58.342702 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.343398 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.345443 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.346383 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.346555 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.346636 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:58.347090 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 19:41:58.347238 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 19:41:58.347405 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.340018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:44779 (local address 127.21.202.65:57103)
0806 19:41:58.341308 (+ 1290us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:58.341348 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:58.341520 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:58.341912 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:58.341916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:58.341941 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:41:58.342370 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:58.342381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.343537 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.343543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:58.345118 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:58.345130 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.346510 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.346517 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:58.346709 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.346777 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 19:41:58.346882 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 19:41:58.346986 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1174,"thread_start_us":928,"threads_started":1}
I20260806 19:41:58.347556 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.339886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:38213 (local address 127.21.202.65:51535)
0806 19:41:58.341216 (+ 1330us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:58.341234 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:58.341471 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:58.341966 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:58.341971 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:58.342001 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:41:58.342543 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:58.342558 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.343547 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.343554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:58.345270 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:58.345285 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.346700 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.346709 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:58.346900 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.346933 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 19:41:58.347031 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 19:41:58.347153 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1168,"mutex_wait_us":5,"thread_start_us":692,"threads_started":1}
I20260806 19:41:58.349062 23168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd4723400f8b4ef99b4246a665f482dd" candidate_uuid: "84773e3a70164836b1ded49427541bba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "789f1fb0206b46cd81432fbb763eedde" is_pre_election: true
I20260806 19:41:58.349098 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd4723400f8b4ef99b4246a665f482dd" candidate_uuid: "84773e3a70164836b1ded49427541bba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8f31eae9c854632ae62de37142b1b18" is_pre_election: true
I20260806 19:41:58.349941 23168 raft_consensus.cc:2454] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84773e3a70164836b1ded49427541bba in term 0.
I20260806 19:41:58.350767 22967 leader_election.cc:302] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 789f1fb0206b46cd81432fbb763eedde, 84773e3a70164836b1ded49427541bba; no voters:
I20260806 19:41:58.351183 23373 raft_consensus.cc:2781] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:41:58.351150 23302 raft_consensus.cc:2454] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84773e3a70164836b1ded49427541bba in term 0.
I20260806 19:41:58.351311 23373 raft_consensus.cc:482] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:41:58.351428 23373 raft_consensus.cc:3037] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:58.353773 23373 raft_consensus.cc:504] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84773e3a70164836b1ded49427541bba" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 45063 } } peers { permanent_uuid: "789f1fb0206b46cd81432fbb763eedde" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38213 } } peers { permanent_uuid: "f8f31eae9c854632ae62de37142b1b18" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44779 } }
I20260806 19:41:58.354468 23373 leader_election.cc:288] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [CANDIDATE]: Term 1 election: Requested vote from peers 789f1fb0206b46cd81432fbb763eedde (127.21.202.66:38213), f8f31eae9c854632ae62de37142b1b18 (127.21.202.67:44779)
I20260806 19:41:58.354911 23168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd4723400f8b4ef99b4246a665f482dd" candidate_uuid: "84773e3a70164836b1ded49427541bba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "789f1fb0206b46cd81432fbb763eedde"
I20260806 19:41:58.355000 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd4723400f8b4ef99b4246a665f482dd" candidate_uuid: "84773e3a70164836b1ded49427541bba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8f31eae9c854632ae62de37142b1b18"
I20260806 19:41:58.355134 23168 raft_consensus.cc:3037] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:58.355237 23302 raft_consensus.cc:3037] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18 [term 0 FOLLOWER]: Advancing to term 1
W20260806 19:41:58.357415 23348 tablet.cc:2369] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:41:58.357822 23168 raft_consensus.cc:2454] T dd4723400f8b4ef99b4246a665f482dd P 789f1fb0206b46cd81432fbb763eedde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84773e3a70164836b1ded49427541bba in term 1.
I20260806 19:41:58.357813 23302 raft_consensus.cc:2454] T dd4723400f8b4ef99b4246a665f482dd P f8f31eae9c854632ae62de37142b1b18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84773e3a70164836b1ded49427541bba in term 1.
I20260806 19:41:58.358335 22968 leader_election.cc:302] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84773e3a70164836b1ded49427541bba, f8f31eae9c854632ae62de37142b1b18; no voters:
I20260806 19:41:58.358723 23373 raft_consensus.cc:2781] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:58.359184 23373 raft_consensus.cc:686] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [term 1 LEADER]: Becoming Leader. State: Replica: 84773e3a70164836b1ded49427541bba, State: Running, Role: LEADER
I20260806 19:41:58.359668 23373 consensus_queue.cc:237] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84773e3a70164836b1ded49427541bba" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 45063 } } peers { permanent_uuid: "789f1fb0206b46cd81432fbb763eedde" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38213 } } peers { permanent_uuid: "f8f31eae9c854632ae62de37142b1b18" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44779 } }
I20260806 19:41:58.365407 22889 catalog_manager.cc:5495] T dd4723400f8b4ef99b4246a665f482dd P 84773e3a70164836b1ded49427541bba reported cstate change: term changed from 0 to 1, leader changed from <none> to 84773e3a70164836b1ded49427541bba (127.21.202.65). New cstate: current_term: 1 leader_uuid: "84773e3a70164836b1ded49427541bba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84773e3a70164836b1ded49427541bba" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 45063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "789f1fb0206b46cd81432fbb763eedde" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8f31eae9c854632ae62de37142b1b18" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44779 } health_report { overall_health: UNKNOWN } } }
Aug 06 19:41:58 dist-test-slave-77v9 krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:41:58.409469 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.390711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33080 (local address 127.21.202.65:45063)
0806 19:41:58.391019 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:58.391028 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:58.391574 (+ 546us) server_negotiation.cc:407] Connection header received
0806 19:41:58.391832 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:58.391842 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:58.391925 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:58.392057 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:58.394212 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.395102 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.396464 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.396899 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.397018 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.402115 (+ 5097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:58.402136 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:58.402153 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:58.402197 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:58.405753 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.406718 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.406726 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.406728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.406942 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.407472 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.407476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.407478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.407926 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:58.408178 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:58.408622 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 19:41:58.408946 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
Aug 06 19:41:58 dist-test-slave-77v9 krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045318, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 19:41:58 dist-test-slave-77v9 krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045318, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:41:58.466495 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.449892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33094 (local address 127.21.202.65:45063)
0806 19:41:58.450123 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:58.450133 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:58.450581 (+ 448us) server_negotiation.cc:407] Connection header received
0806 19:41:58.450825 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:58.450829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:58.450897 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:58.451013 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:58.452407 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.453255 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.454557 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.455005 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.455147 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.459613 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:58.459619 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:58.459623 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:58.459701 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:58.463120 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.463937 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.463950 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.464016 (+ 66us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.464219 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.464743 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.464750 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.464753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.465157 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:58.465514 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:58.465818 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 19:41:58.466088 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 19:41:58 dist-test-slave-77v9 krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 19:41:58 dist-test-slave-77v9 krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:41:58.519685 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.504591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.21.202.65:45063)
0806 19:41:58.504852 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:58.504860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:58.505207 (+ 347us) server_negotiation.cc:407] Connection header received
0806 19:41:58.505434 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:58.505439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:58.505520 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:58.505643 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:58.507117 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.508063 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.509397 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.509816 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.509961 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.514140 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:58.514145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:58.514148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:58.514186 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:58.516914 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.517610 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.517627 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.517631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.517770 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.518180 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.518184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.518186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.518479 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:58.518758 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:58.519064 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 19:41:58.519293 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260806 19:41:58.522233 22994 tablet_service.cc:2908] Rejecting scan request for tablet dd4723400f8b4ef99b4246a665f482dd: Uninitialized: clean time has not yet been initialized
I20260806 19:41:58.526418 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 22951
I20260806 19:41:58.541584 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.527553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33104 (local address 127.21.202.65:45063)
0806 19:41:58.527778 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:58.527786 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:58.528250 (+ 464us) server_negotiation.cc:407] Connection header received
0806 19:41:58.528430 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:58.528434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:58.528498 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:58.528591 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:58.529916 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.530858 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.532154 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.532577 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.532673 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.535106 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:58.535112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:58.535116 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:58.535152 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:58.538384 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.539268 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.539272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.539275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.539429 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.539911 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.539917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.539920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.540262 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:58.540497 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:58.540968 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 19:41:58.541244 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 19:41:58.542402 23074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:58.639700 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 22951
I20260806 19:41:58.658430 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 23085
Aug 06 19:41:58 dist-test-slave-77v9 krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:41:58.675756 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.659342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54740 (local address 127.21.202.66:38213)
0806 19:41:58.659559 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:58.659566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:58.659674 (+ 108us) server_negotiation.cc:407] Connection header received
0806 19:41:58.659959 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:58.659965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:58.660058 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:58.660192 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:58.661611 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.662541 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.663972 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.664391 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.664496 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.669023 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:58.669042 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:58.669060 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:58.669102 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:58.672299 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.673465 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.673474 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.673476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.673618 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.674050 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.674054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.674057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.674473 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:58.674842 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:58.675144 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 19:41:58.675389 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 19:41:58.676553 23208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:58.767257 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 23085
I20260806 19:41:58.787130 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 23219
Aug 06 19:41:58 dist-test-slave-77v9 krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:41:58.803825 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:58.788026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59526 (local address 127.21.202.67:44779)
0806 19:41:58.788229 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:58.788233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:58.788253 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:41:58.788336 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:58.788341 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:58.788407 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:58.788501 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:58.790158 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.790954 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.792433 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:58.792918 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:58.793063 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:58.797770 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:58.797789 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:58.797807 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:58.797861 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:58.800718 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.801555 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.801563 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.801565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.801702 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:58.802185 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:58.802189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:58.802192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:58.802590 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:58.802825 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:58.803211 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 19:41:58.803402 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 19:41:58.804689 23342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:58.895951 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 23219
I20260806 19:41:58.914113 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 22858
I20260806 19:41:58.915318 22920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:59.025373 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 22858
2026-08-06T19:41:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4431 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:59 dist-test-slave-77v9 krb5kdc[23397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:59 dist-test-slave-77v9 krb5kdc[23397](info): set up 2 sockets
Aug 06 19:41:59 dist-test-slave-77v9 krb5kdc[23397](info): commencing operation
krb5kdc: starting...
W20260806 19:42:00.230535 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:42:00 dist-test-slave-77v9 krb5kdc[23397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:42:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:42:00.459095 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:43777
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:43777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:00.812330 23410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:00.812681 23410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:00.812872 23410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:00.820693 23410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:00.820880 23410 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 19:42:00.820982 23410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:00.821097 23410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:00.821175 23410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:00.821277 23410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:42:00.832031 23410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:43777
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:43777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:00.833532 23410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:00.835003 23410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:00.848014 23417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:00.848052 23415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:00.848088 23419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:00.848974 23410 server_base.cc:1034] running on GCE node
I20260806 19:42:00.849942 23410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:00.851502 23410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:00.852778 23410 hybrid_clock.cc:648] HybridClock initialized: now 1786045320852679 us; error 55 us; skew 500 ppm
I20260806 19:42:00.853248 23410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:00 dist-test-slave-77v9 krb5kdc[23397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045320, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:00.857165 23410 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:42:00.859005 23410 webserver.cc:460] Webserver started at http://127.21.202.126:36031/ using document root <none> and password file <none>
I20260806 19:42:00.859800 23410 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:00.859966 23410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:00.860371 23410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:00.863598 23410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "f78567e110ef4edfa792340b7921161e"
format_stamp: "Formatted at 2026-08-06 19:42:00 on dist-test-slave-77v9"
server_key: "95c790d8fdbd8bbd68e54c040b52e561"
server_key_iv: "0d40882c5765e888dd77c1de68bd2445"
server_key_version: "encryptionkey@0"
I20260806 19:42:00.864482 23410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "f78567e110ef4edfa792340b7921161e"
format_stamp: "Formatted at 2026-08-06 19:42:00 on dist-test-slave-77v9"
server_key: "95c790d8fdbd8bbd68e54c040b52e561"
server_key_iv: "0d40882c5765e888dd77c1de68bd2445"
server_key_version: "encryptionkey@0"
I20260806 19:42:00.870286 23410 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 19:42:00.873961 23426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:00.875500 23410 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:42:00.875705 23410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "f78567e110ef4edfa792340b7921161e"
format_stamp: "Formatted at 2026-08-06 19:42:00 on dist-test-slave-77v9"
server_key: "95c790d8fdbd8bbd68e54c040b52e561"
server_key_iv: "0d40882c5765e888dd77c1de68bd2445"
server_key_version: "encryptionkey@0"
I20260806 19:42:00.875965 23410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:00.895416 23410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:00.900336 23410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:00.900705 23410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:00.926138 23410 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:43777
I20260806 19:42:00.926158 23487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:43777 every 8 connection(s)
I20260806 19:42:00.927685 23410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:00.931736 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 23410
I20260806 19:42:00.931957 23488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:00.932060 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:00.932881 22313 external_mini_cluster.cc:1428] Setting key bfedbaf2d797a19742cf662e2178cf4b
I20260806 19:42:00.948973 23488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e: Bootstrap starting.
Aug 06 19:42:00 dist-test-slave-77v9 krb5kdc[23397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:00.952418 23488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:00.953853 23488 log.cc:824] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:00.958068 23488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e: No bootstrap required, opened a new log
I20260806 19:42:00.962586 23492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:00.935414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33846 (local address 127.21.202.126:43777)
0806 19:42:00.936382 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:00.936403 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:00.936462 (+ 59us) server_negotiation.cc:407] Connection header received
0806 19:42:00.939807 (+ 3345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:00.939846 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:00.940517 (+ 671us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:00.941111 (+ 594us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:00.942500 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:00.944777 (+ 2277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:00.946332 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:00.947520 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:00.947729 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:00.952395 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:00.952420 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:00.952442 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:00.952505 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:00.956721 (+ 4216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:00.957613 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:00.957623 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:00.957632 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:00.957899 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:00.958370 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:00.958376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:00.958379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:00.959251 (+ 872us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:00.959579 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:00.960264 (+ 685us) server_negotiation.cc:299] Negotiation successful
0806 19:42:00.960937 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":420,"threads_started":1}
I20260806 19:42:00.968775 23488 raft_consensus.cc:348] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f78567e110ef4edfa792340b7921161e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 43777 } }
I20260806 19:42:00.969270 23488 raft_consensus.cc:374] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:00.969357 23488 raft_consensus.cc:729] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f78567e110ef4edfa792340b7921161e, State: Initialized, Role: FOLLOWER
I20260806 19:42:00.969883 23488 consensus_queue.cc:260] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f78567e110ef4edfa792340b7921161e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 43777 } }
I20260806 19:42:00.970161 23488 raft_consensus.cc:388] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:00.970348 23488 raft_consensus.cc:482] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:00.970567 23488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:00.973040 23488 raft_consensus.cc:504] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f78567e110ef4edfa792340b7921161e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 43777 } }
I20260806 19:42:00.973624 23488 leader_election.cc:302] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f78567e110ef4edfa792340b7921161e; no voters:
I20260806 19:42:00.974339 23488 leader_election.cc:288] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:00.974505 23494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:00.975919 23494 raft_consensus.cc:686] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [term 1 LEADER]: Becoming Leader. State: Replica: f78567e110ef4edfa792340b7921161e, State: Running, Role: LEADER
I20260806 19:42:00.976423 23494 consensus_queue.cc:237] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f78567e110ef4edfa792340b7921161e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 43777 } }
I20260806 19:42:00.976773 23488 sys_catalog.cc:564] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:00.979352 23496 sys_catalog.cc:455] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f78567e110ef4edfa792340b7921161e. Latest consensus state: current_term: 1 leader_uuid: "f78567e110ef4edfa792340b7921161e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f78567e110ef4edfa792340b7921161e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 43777 } } }
I20260806 19:42:00.979714 23496 sys_catalog.cc:458] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:00.980280 23495 sys_catalog.cc:455] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f78567e110ef4edfa792340b7921161e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f78567e110ef4edfa792340b7921161e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 43777 } } }
I20260806 19:42:00.980580 23495 sys_catalog.cc:458] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:00.981266 23501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:00.991225 23501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:01.000626 23501 catalog_manager.cc:1347] Generated new cluster ID: 7a11096cf0764cb0bd539a7eb26f99d7
I20260806 19:42:01.000779 23501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:01.020519 23501 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:01.022202 23501 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:01.031267 23501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e: Generated new TSK 0
I20260806 19:42:01.032248 23501 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:42:01.100699 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:43777
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:01.455777 23515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:01.456076 23515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:01.456175 23515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:01.464054 23515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:01.464183 23515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:01.464277 23515 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:42:01.464362 23515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:42:01.474872 23515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.202.126:43777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:01.476338 23515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:01.477903 23515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:01.491817 23520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:01.491817 23523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:01.491845 23521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:01.493490 23515 server_base.cc:1034] running on GCE node
I20260806 19:42:01.494177 23515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:01.495193 23515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:01.496471 23515 hybrid_clock.cc:648] HybridClock initialized: now 1786045321496378 us; error 62 us; skew 500 ppm
I20260806 19:42:01.496897 23515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:01 dist-test-slave-77v9 krb5kdc[23397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045321, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:01.500245 23515 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:42:01.502071 23515 webserver.cc:460] Webserver started at http://127.21.202.65:42767/ using document root <none> and password file <none>
I20260806 19:42:01.502848 23515 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:01.502993 23515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:01.503376 23515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:01.506740 23515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "a5b742f4edd0459ca802bcc95f1bb2ea"
format_stamp: "Formatted at 2026-08-06 19:42:01 on dist-test-slave-77v9"
server_key: "b2f78a3345efd682b95f89dcbcacb86b"
server_key_iv: "518323f807263c689f96cfdecfd2e0eb"
server_key_version: "encryptionkey@0"
I20260806 19:42:01.507652 23515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "a5b742f4edd0459ca802bcc95f1bb2ea"
format_stamp: "Formatted at 2026-08-06 19:42:01 on dist-test-slave-77v9"
server_key: "b2f78a3345efd682b95f89dcbcacb86b"
server_key_iv: "518323f807263c689f96cfdecfd2e0eb"
server_key_version: "encryptionkey@0"
I20260806 19:42:01.513712 23515 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 19:42:01.517588 23530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:01.519250 23515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:42:01.519466 23515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "a5b742f4edd0459ca802bcc95f1bb2ea"
format_stamp: "Formatted at 2026-08-06 19:42:01 on dist-test-slave-77v9"
server_key: "b2f78a3345efd682b95f89dcbcacb86b"
server_key_iv: "518323f807263c689f96cfdecfd2e0eb"
server_key_version: "encryptionkey@0"
I20260806 19:42:01.519799 23515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:01.544059 23515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:01.548969 23515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:01.549296 23515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:01.552397 23515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:01.552528 23515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:01.552693 23515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:01.552791 23515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:42:01 dist-test-slave-77v9 krb5kdc[23397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045321, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:01.578576 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:01.553211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:52429 (local address 127.21.202.126:43777)
0806 19:42:01.553889 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:01.553896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:01.555141 (+ 1245us) server_negotiation.cc:407] Connection header received
0806 19:42:01.558686 (+ 3545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:01.558692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:01.558794 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:01.558937 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:01.561250 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:01.562296 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:01.564943 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:01.565351 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:01.565487 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:01.570851 (+ 5364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:01.570860 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:01.570864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:01.570903 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:01.574117 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:01.575226 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:01.575232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:01.575236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:01.575367 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:01.576070 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:01.576075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:01.576079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:01.576511 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:01.576685 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:01.577917 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0806 19:42:01.578159 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":411,"threads_started":1}
I20260806 19:42:01.579799 23540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:01.553960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:43777 (local address 127.21.202.65:52429)
0806 19:42:01.554934 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:01.555001 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:01.558251 (+ 3250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:01.559271 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:01.559286 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:01.560140 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:01.561046 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:01.561069 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:01.562505 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:01.562522 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:01.564667 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:01.564681 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:01.565609 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:01.565616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:01.565980 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:01.566868 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:01.566915 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:01.570519 (+ 3604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:01.574284 (+ 3765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:01.574294 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:01.574316 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:01.575074 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:01.575531 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:01.575539 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:01.575546 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:01.575908 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:01.576665 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:01.576673 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:01.577350 (+ 677us) client_negotiation.cc:769] Sending connection context
0806 19:42:01.577781 (+ 431us) client_negotiation.cc:240] Negotiation successful
0806 19:42:01.578401 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":369,"threads_started":1}
I20260806 19:42:01.605901 23515 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:36509
I20260806 19:42:01.605991 23652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:36509 every 8 connection(s)
I20260806 19:42:01.607592 23515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:42:01.610299 23653 heartbeater.cc:347] Connected to a master server at 127.21.202.126:43777
I20260806 19:42:01.611603 23653 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:01.612506 23653 heartbeater.cc:511] Master 127.21.202.126:43777 requested a full tablet report, sending...
I20260806 19:42:01.614282 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:01.596410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:36183 (local address 127.21.202.126:43777)
0806 19:42:01.596664 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:01.596674 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:01.597828 (+ 1154us) server_negotiation.cc:407] Connection header received
0806 19:42:01.598062 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:01.598068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:01.598163 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:01.598323 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:01.599704 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:01.601136 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:01.603916 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:01.604449 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:01.604619 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:01.607288 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:01.607294 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:01.607297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:01.607336 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:01.610867 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:01.611896 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:01.611902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:01.611906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:01.612081 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:01.612504 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:01.612509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:01.612513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:01.612947 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:01.613112 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:01.613711 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 19:42:01.614018 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 19:42:01.614352 23631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:01.596372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:43777 (local address 127.21.202.65:36183)
0806 19:42:01.597624 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:01.597659 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:01.597868 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:01.598375 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:01.598384 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:01.599199 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:01.599537 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:01.599547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:01.601330 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:01.601338 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:01.603731 (+ 2393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:01.603748 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:01.604699 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:01.604710 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:01.604997 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:01.605895 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:01.605920 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:01.607054 (+ 1134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:01.611065 (+ 4011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:01.611073 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:01.611080 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:01.611758 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:01.612224 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:01.612228 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:01.612231 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:01.612380 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:01.613093 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:01.613097 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:01.613279 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 19:42:01.613463 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 19:42:01.613827 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1016,"thread_start_us":748,"threads_started":1}
I20260806 19:42:01.615340 23443 ts_manager.cc:194] Registered new tserver with Master: a5b742f4edd0459ca802bcc95f1bb2ea (127.21.202.65:36509)
I20260806 19:42:01.616271 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 23515
I20260806 19:42:01.616501 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:42:01.617220 22313 external_mini_cluster.cc:1428] Setting key 98dda0196fc5fca89375a3f696869241
I20260806 19:42:01.617414 23443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:52429
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:42:01.668449 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:43777
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:01.999825 23657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:02.000171 23657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:02.000345 23657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:02.007843 23657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:02.007992 23657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:02.008136 23657 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:42:02.008272 23657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:42:02.018800 23657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.202.126:43777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:02.020259 23657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:02.021732 23657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:02.034983 23662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:02.035137 23663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:02.035109 23665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:02.035877 23657 server_base.cc:1034] running on GCE node
I20260806 19:42:02.036580 23657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:02.037878 23657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:02.039139 23657 hybrid_clock.cc:648] HybridClock initialized: now 1786045322039078 us; error 36 us; skew 500 ppm
I20260806 19:42:02.039453 23657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:02 dist-test-slave-77v9 krb5kdc[23397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045322, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:02.042668 23657 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:42:02.044211 23657 webserver.cc:460] Webserver started at http://127.21.202.66:33321/ using document root <none> and password file <none>
I20260806 19:42:02.045001 23657 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:02.045094 23657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:02.045380 23657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:02.048301 23657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "eb861df2948c4ba3a936882765a971da"
format_stamp: "Formatted at 2026-08-06 19:42:02 on dist-test-slave-77v9"
server_key: "7a03883ec9af1352bf636eaa0a768961"
server_key_iv: "48a1e4c01d1e1daddf294de0feb12cb6"
server_key_version: "encryptionkey@0"
I20260806 19:42:02.049125 23657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "eb861df2948c4ba3a936882765a971da"
format_stamp: "Formatted at 2026-08-06 19:42:02 on dist-test-slave-77v9"
server_key: "7a03883ec9af1352bf636eaa0a768961"
server_key_iv: "48a1e4c01d1e1daddf294de0feb12cb6"
server_key_version: "encryptionkey@0"
I20260806 19:42:02.054363 23657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 19:42:02.057832 23672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:02.059378 23657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 19:42:02.059512 23657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "eb861df2948c4ba3a936882765a971da"
format_stamp: "Formatted at 2026-08-06 19:42:02 on dist-test-slave-77v9"
server_key: "7a03883ec9af1352bf636eaa0a768961"
server_key_iv: "48a1e4c01d1e1daddf294de0feb12cb6"
server_key_version: "encryptionkey@0"
I20260806 19:42:02.059689 23657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:02.080942 23657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:02.085966 23657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:02.086225 23657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:02.089509 23657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:02.089632 23657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:02.089787 23657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:02.089861 23657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:42:02 dist-test-slave-77v9 krb5kdc[23397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045322, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:02.118712 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.090439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:47573 (local address 127.21.202.126:43777)
0806 19:42:02.090664 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:02.090672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:02.092892 (+ 2220us) server_negotiation.cc:407] Connection header received
0806 19:42:02.095342 (+ 2450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:02.095352 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:02.095475 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:02.095601 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:02.098162 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.099183 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.102814 (+ 3631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.103542 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.103688 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.109945 (+ 6257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:02.109955 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:02.109962 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:02.110026 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:02.114121 (+ 4095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:02.115307 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:02.115314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:02.115318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:02.115521 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:02.116203 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:02.116210 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:02.116215 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:02.116700 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:02.116919 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:02.117809 (+ 890us) server_negotiation.cc:299] Negotiation successful
0806 19:42:02.118138 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 19:42:02.119364 23681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.091329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:43777 (local address 127.21.202.66:47573)
0806 19:42:02.092484 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:02.092632 (+ 148us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:02.094854 (+ 2222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:02.095945 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:02.095968 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:02.096893 (+ 925us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:02.097938 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.097972 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.099408 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.099417 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.102599 (+ 3182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.102628 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.103844 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.103854 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.104104 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.104884 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:02.104923 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:02.109539 (+ 4616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:02.114341 (+ 4802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:02.114353 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:02.114379 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:02.115148 (+ 769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:02.115704 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:02.115709 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:02.115713 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:02.116028 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:02.116931 (+ 903us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:02.116945 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:02.117355 (+ 410us) client_negotiation.cc:769] Sending connection context
0806 19:42:02.117612 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 19:42:02.118041 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"thread_start_us":574,"threads_started":1}
I20260806 19:42:02.143631 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.126512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:38861 (local address 127.21.202.126:43777)
0806 19:42:02.126718 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:02.126725 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:02.127209 (+ 484us) server_negotiation.cc:407] Connection header received
0806 19:42:02.127414 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:02.127419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:02.127494 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:02.127596 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:02.128560 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.129550 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.131420 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.132031 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.132176 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.135168 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:02.135173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:02.135177 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:02.135229 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:02.139274 (+ 4045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:02.140850 (+ 1576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:02.140859 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:02.140866 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:02.141179 (+ 313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:02.141909 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:02.141914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:02.141917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:02.142400 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:02.142551 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:02.143175 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 19:42:02.143378 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 19:42:02.144270 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.126437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:43777 (local address 127.21.202.66:38861)
0806 19:42:02.127093 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:02.127113 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:02.127281 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:02.127623 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:02.127628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:02.128113 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:02.128413 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.128423 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.129703 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.129716 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.131187 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.131200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.132295 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.132306 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.132572 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.133465 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:02.133505 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:02.134925 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:02.139551 (+ 4626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:02.139563 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:02.139570 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:02.140618 (+ 1048us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:02.141392 (+ 774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:02.141403 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:02.141410 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:02.141683 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:02.142553 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:02.142560 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:02.142854 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 19:42:02.143058 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 19:42:02.143570 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":364,"threads_started":1}
I20260806 19:42:02.154409 23657 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:32883
I20260806 19:42:02.154506 23792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:32883 every 8 connection(s)
I20260806 19:42:02.155689 23657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:42:02.157966 23793 heartbeater.cc:347] Connected to a master server at 127.21.202.126:43777
I20260806 19:42:02.158200 23793 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:02.158878 23793 heartbeater.cc:511] Master 127.21.202.126:43777 requested a full tablet report, sending...
I20260806 19:42:02.160344 23443 ts_manager.cc:194] Registered new tserver with Master: eb861df2948c4ba3a936882765a971da (127.21.202.66:32883)
I20260806 19:42:02.161211 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 23657
I20260806 19:42:02.161415 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:42:02.161375 23443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:47573
I20260806 19:42:02.161880 22313 external_mini_cluster.cc:1428] Setting key 5029a214e385397895494480205ca34b
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:42:02.213384 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:43777
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:02.558409 23797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:02.558736 23797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:02.558903 23797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:02.566612 23797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:02.566776 23797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:02.566917 23797 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:42:02.567049 23797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:42:02.577292 23797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.202.126:43777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:02.578727 23797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:02.580217 23797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:02.593917 23803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:02.594162 23805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:02.594727 23802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:02.595557 23797 server_base.cc:1034] running on GCE node
I20260806 19:42:02.596354 23797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:02.597438 23797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:02.598712 23797 hybrid_clock.cc:648] HybridClock initialized: now 1786045322598630 us; error 53 us; skew 500 ppm
I20260806 19:42:02.599056 23797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:02 dist-test-slave-77v9 krb5kdc[23397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045322, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:02.606801 23797 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:42:02.608378 23797 webserver.cc:460] Webserver started at http://127.21.202.67:41103/ using document root <none> and password file <none>
I20260806 19:42:02.609136 23797 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:02.609248 23797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:02.609591 23797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:02.612495 23797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "18aace29b43a490e8c2b92d457c2eaa9"
format_stamp: "Formatted at 2026-08-06 19:42:02 on dist-test-slave-77v9"
server_key: "cac89e5fadfa019eec64a18be16b4b33"
server_key_iv: "1ed9fc21cefc4843204cc4ef2c7dc473"
server_key_version: "encryptionkey@0"
I20260806 19:42:02.613337 23797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "18aace29b43a490e8c2b92d457c2eaa9"
format_stamp: "Formatted at 2026-08-06 19:42:02 on dist-test-slave-77v9"
server_key: "cac89e5fadfa019eec64a18be16b4b33"
server_key_iv: "1ed9fc21cefc4843204cc4ef2c7dc473"
server_key_version: "encryptionkey@0"
I20260806 19:42:02.619546 23797 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260806 19:42:02.620537 23653 heartbeater.cc:503] Master 127.21.202.126:43777 was elected leader, sending a full tablet report...
I20260806 19:42:02.623508 23812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:02.625154 23797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 19:42:02.625324 23797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "18aace29b43a490e8c2b92d457c2eaa9"
format_stamp: "Formatted at 2026-08-06 19:42:02 on dist-test-slave-77v9"
server_key: "cac89e5fadfa019eec64a18be16b4b33"
server_key_iv: "1ed9fc21cefc4843204cc4ef2c7dc473"
server_key_version: "encryptionkey@0"
I20260806 19:42:02.625569 23797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:02.654539 23797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:02.659374 23797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:02.659673 23797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:02.662634 23797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:02.662734 23797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 19:42:02.662883 23797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:02.662982 23797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:42:02 dist-test-slave-77v9 krb5kdc[23397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045322, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:02.686786 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.663286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:37711 (local address 127.21.202.126:43777)
0806 19:42:02.664134 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:02.664138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:02.665137 (+ 999us) server_negotiation.cc:407] Connection header received
0806 19:42:02.667214 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:02.667220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:02.667323 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:02.667448 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:02.669935 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.671043 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.673008 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.673631 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.673768 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.679001 (+ 5233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:02.679007 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:02.679010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:02.679048 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:02.682820 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:02.684052 (+ 1232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:02.684058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:02.684061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:02.684208 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:02.684686 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:02.684691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:02.684694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:02.685070 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:02.685242 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:02.686109 (+ 867us) server_negotiation.cc:299] Negotiation successful
0806 19:42:02.686341 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":608,"threads_started":1}
I20260806 19:42:02.687913 23822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.663995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:43777 (local address 127.21.202.67:37711)
0806 19:42:02.664947 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:02.665013 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:02.666916 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:02.667723 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:02.667731 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:02.668564 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:02.669711 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.669746 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.671216 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.671224 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.672782 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.672856 (+ 74us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.673894 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.673902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.674202 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.674950 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:02.674973 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:02.678685 (+ 3712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:02.682988 (+ 4303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:02.682998 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:02.683022 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:02.683903 (+ 881us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:02.684332 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:02.684337 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:02.684341 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:02.684559 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:02.685213 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:02.685222 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:02.685658 (+ 436us) client_negotiation.cc:769] Sending connection context
0806 19:42:02.685988 (+ 330us) client_negotiation.cc:240] Negotiation successful
0806 19:42:02.686583 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":397,"threads_started":1}
I20260806 19:42:02.716183 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.695034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:49203 (local address 127.21.202.126:43777)
0806 19:42:02.695242 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:02.695248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:02.697527 (+ 2279us) server_negotiation.cc:407] Connection header received
0806 19:42:02.697671 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:02.697680 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:02.697804 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:02.697932 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:02.699733 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.701080 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.704004 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.704691 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.704910 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.707894 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:02.707899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:02.707903 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:02.707949 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:02.712318 (+ 4369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:02.713596 (+ 1278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:02.713602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:02.713605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:02.713740 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:02.714228 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:02.714234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:02.714239 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:02.714717 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:02.714929 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:02.715613 (+ 684us) server_negotiation.cc:299] Negotiation successful
0806 19:42:02.715881 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 19:42:02.716495 23881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.695332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:43777 (local address 127.21.202.67:49203)
0806 19:42:02.697385 (+ 2053us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:02.697407 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:02.697554 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:02.698000 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:02.698008 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:02.698934 (+ 926us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:02.699504 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.699524 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.701333 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.701339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.703804 (+ 2465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.703828 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.704992 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.705006 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.705243 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.706128 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:02.706157 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:02.707659 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:02.712543 (+ 4884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:02.712552 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:02.712560 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:02.713433 (+ 873us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:02.713876 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:02.713884 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:02.713890 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:02.714087 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:02.714913 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:02.714921 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:02.715302 (+ 381us) client_negotiation.cc:769] Sending connection context
0806 19:42:02.715527 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 19:42:02.715909 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1840,"thread_start_us":1650,"threads_started":1}
I20260806 19:42:02.717937 23797 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:37343
I20260806 19:42:02.718058 23933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:37343 every 8 connection(s)
I20260806 19:42:02.719599 23797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:42:02.721828 23934 heartbeater.cc:347] Connected to a master server at 127.21.202.126:43777
I20260806 19:42:02.722059 23934 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:02.722841 23934 heartbeater.cc:511] Master 127.21.202.126:43777 requested a full tablet report, sending...
I20260806 19:42:02.724318 23443 ts_manager.cc:194] Registered new tserver with Master: 18aace29b43a490e8c2b92d457c2eaa9 (127.21.202.67:37343)
I20260806 19:42:02.725296 23443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:37711
I20260806 19:42:02.726436 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 23797
I20260806 19:42:02.726660 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:42:02.727180 22313 external_mini_cluster.cc:1428] Setting key e0e2b47587d02bb4c64e8ba1cb416119
I20260806 19:42:02.730684 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 19:42:02 dist-test-slave-77v9 krb5kdc[23397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045322, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 19:42:02 dist-test-slave-77v9 krb5kdc[23397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045322, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:02.785032 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.769031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33858 (local address 127.21.202.126:43777)
0806 19:42:02.769209 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:02.769214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:02.769655 (+ 441us) server_negotiation.cc:407] Connection header received
0806 19:42:02.769815 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:02.769820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:02.769914 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:02.770034 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:02.771676 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.772594 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.774144 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.774577 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.774693 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.779121 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:02.779127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:02.779130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:02.779169 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:02.782110 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:02.782937 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:02.782941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:02.782944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:02.783095 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:02.783578 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:02.783582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:02.783584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:02.783922 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:02.784132 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:02.784471 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 19:42:02.784659 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 19:42:02.790177 23443 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:42:02.798444 23443 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:42:02.829900 23950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.811936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:32883 (local address 127.0.0.1:50350)
0806 19:42:02.819862 (+ 7926us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:02.819893 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:02.820024 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:02.821556 (+ 1532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:02.821564 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:02.821599 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:02.822046 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.822065 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.823961 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.823972 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.826392 (+ 2420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.826411 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.828756 (+ 2345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.828763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.829020 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.829077 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 19:42:02.829225 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 19:42:02.829430 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7740,"mutex_wait_us":4,"thread_start_us":6515,"threads_started":1}
I20260806 19:42:02.829888 23951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.812494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50350 (local address 127.21.202.66:32883)
0806 19:42:02.820542 (+ 8048us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:02.820556 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:02.820592 (+ 36us) server_negotiation.cc:407] Connection header received
0806 19:42:02.820710 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:02.820720 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:02.821022 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:02.821240 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:02.822372 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.823798 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.826600 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.828193 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.828353 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.828620 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:02.829302 (+ 682us) server_negotiation.cc:299] Negotiation successful
0806 19:42:02.829511 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7814,"thread_start_us":7675,"threads_started":1}
I20260806 19:42:02.831713 23948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.812404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:36509 (local address 127.0.0.1:38626)
0806 19:42:02.822448 (+ 10044us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:02.822481 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:02.822634 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:02.823256 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:02.823261 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:02.823289 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:02.823715 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.823732 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.825652 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.825659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.828198 (+ 2539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.828213 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.830663 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.830668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.830950 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.830985 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 19:42:02.831094 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 19:42:02.831241 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9908,"mutex_wait_us":190,"thread_start_us":7127,"threads_started":1}
I20260806 19:42:02.831734 23947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.811861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:37343 (local address 127.0.0.1:41442)
0806 19:42:02.818873 (+ 7012us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:02.818934 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:02.819129 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:02.821827 (+ 2698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:02.821831 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:02.821898 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:02.822644 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.822654 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.825360 (+ 2706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.825367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.827947 (+ 2580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.827962 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.830852 (+ 2890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.830866 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.831128 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.831155 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 19:42:02.831230 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 19:42:02.831348 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6746,"mutex_wait_us":213,"thread_start_us":10208,"threads_started":1}
I20260806 19:42:02.832291 23949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.812179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41442 (local address 127.21.202.67:37343)
0806 19:42:02.821070 (+ 8891us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:02.821080 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:02.821110 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:42:02.821210 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:02.821216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:02.821547 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:02.822251 (+ 704us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:02.822864 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.825199 (+ 2335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.828190 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.830633 (+ 2443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.830781 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.831041 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:02.831437 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 19:42:02.831635 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8702,"thread_start_us":8479,"threads_started":1}
I20260806 19:42:02.832540 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.813166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38626 (local address 127.21.202.65:36509)
0806 19:42:02.821835 (+ 8669us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:02.821844 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:02.822588 (+ 744us) server_negotiation.cc:407] Connection header received
0806 19:42:02.822838 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:02.822846 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:02.823071 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:02.823255 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:02.824047 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.825518 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.828416 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.829955 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.830398 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.830556 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:02.832084 (+ 1528us) server_negotiation.cc:299] Negotiation successful
0806 19:42:02.832212 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8528,"thread_start_us":8379,"threads_started":1}
I20260806 19:42:02.835261 23864 tablet_service.cc:1467] Processing CreateTablet for tablet 3835c4caee734c068f54973dc166599d (DEFAULT_TABLE table=test-table [id=7c5e275a71e7425099872993af77965a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:02.835140 23723 tablet_service.cc:1467] Processing CreateTablet for tablet 3835c4caee734c068f54973dc166599d (DEFAULT_TABLE table=test-table [id=7c5e275a71e7425099872993af77965a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:02.836568 23864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3835c4caee734c068f54973dc166599d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:02.836105 23583 tablet_service.cc:1467] Processing CreateTablet for tablet 3835c4caee734c068f54973dc166599d (DEFAULT_TABLE table=test-table [id=7c5e275a71e7425099872993af77965a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:02.836706 23723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3835c4caee734c068f54973dc166599d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:02.837880 23583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3835c4caee734c068f54973dc166599d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:02.848987 23953 tablet_bootstrap.cc:492] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9: Bootstrap starting.
I20260806 19:42:02.852972 23953 tablet_bootstrap.cc:654] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:02.854050 23953 log.cc:824] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:02.856983 23954 tablet_bootstrap.cc:492] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da: Bootstrap starting.
I20260806 19:42:02.857085 23953 tablet_bootstrap.cc:492] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9: No bootstrap required, opened a new log
I20260806 19:42:02.857355 23953 ts_tablet_manager.cc:1397] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 19:42:02.860615 23954 tablet_bootstrap.cc:654] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:02.861574 23955 tablet_bootstrap.cc:492] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea: Bootstrap starting.
I20260806 19:42:02.865307 23955 tablet_bootstrap.cc:654] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:02.866066 23954 log.cc:824] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:02.866690 23955 log.cc:824] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:02.868950 23953 raft_consensus.cc:348] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } }
I20260806 19:42:02.869459 23953 raft_consensus.cc:374] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:02.869586 23953 raft_consensus.cc:729] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18aace29b43a490e8c2b92d457c2eaa9, State: Initialized, Role: FOLLOWER
I20260806 19:42:02.869861 23955 tablet_bootstrap.cc:492] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea: No bootstrap required, opened a new log
I20260806 19:42:02.870029 23954 tablet_bootstrap.cc:492] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da: No bootstrap required, opened a new log
I20260806 19:42:02.870222 23955 ts_tablet_manager.cc:1397] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 19:42:02.870321 23954 ts_tablet_manager.cc:1397] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260806 19:42:02.870416 23953 consensus_queue.cc:260] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } }
I20260806 19:42:02.877411 23934 heartbeater.cc:503] Master 127.21.202.126:43777 was elected leader, sending a full tablet report...
I20260806 19:42:02.879004 23953 ts_tablet_manager.cc:1428] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9: Time spent starting tablet: real 0.022s user 0.020s sys 0.000s
I20260806 19:42:02.879504 23955 raft_consensus.cc:348] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } }
I20260806 19:42:02.879915 23955 raft_consensus.cc:374] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:02.880045 23955 raft_consensus.cc:729] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5b742f4edd0459ca802bcc95f1bb2ea, State: Initialized, Role: FOLLOWER
I20260806 19:42:02.880697 23955 consensus_queue.cc:260] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } }
I20260806 19:42:02.882637 23954 raft_consensus.cc:348] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } }
I20260806 19:42:02.883077 23954 raft_consensus.cc:374] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:02.883140 23955 ts_tablet_manager.cc:1428] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260806 19:42:02.883193 23954 raft_consensus.cc:729] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb861df2948c4ba3a936882765a971da, State: Initialized, Role: FOLLOWER
I20260806 19:42:02.883813 23954 consensus_queue.cc:260] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } }
I20260806 19:42:02.885573 23793 heartbeater.cc:503] Master 127.21.202.126:43777 was elected leader, sending a full tablet report...
I20260806 19:42:02.887272 23954 ts_tablet_manager.cc:1428] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da: Time spent starting tablet: real 0.017s user 0.014s sys 0.003s
W20260806 19:42:02.907872 23794 tablet.cc:2369] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:42:02.942350 23960 raft_consensus.cc:482] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:02.942688 23960 raft_consensus.cc:504] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } }
I20260806 19:42:02.944442 23960 leader_election.cc:288] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18aace29b43a490e8c2b92d457c2eaa9 (127.21.202.67:37343), eb861df2948c4ba3a936882765a971da (127.21.202.66:32883)
I20260806 19:42:02.955245 23949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.945465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:49193 (local address 127.21.202.67:37343)
0806 19:42:02.945724 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:02.945733 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:02.946537 (+ 804us) server_negotiation.cc:407] Connection header received
0806 19:42:02.946897 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:02.946908 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:02.946993 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:02.947101 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:02.948006 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.949572 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.952388 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.954037 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.954287 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.954387 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:02.954632 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 19:42:02.954792 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 19:42:02.955243 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.944678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:37343 (local address 127.21.202.65:49193)
0806 19:42:02.946391 (+ 1713us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:02.946416 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:02.946665 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:02.947156 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:02.947161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:02.947189 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:02.947775 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.947804 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.949768 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.949773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.952167 (+ 2394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.952188 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.954298 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.954304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.954440 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.954467 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 19:42:02.954576 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 19:42:02.954690 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1551,"mutex_wait_us":25,"thread_start_us":1744,"threads_started":1}
I20260806 19:42:02.955853 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.944802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:32883 (local address 127.21.202.65:47133)
0806 19:42:02.946698 (+ 1896us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:02.946715 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:02.946835 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:02.947156 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:02.947161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:02.947217 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:02.947814 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.947827 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.950113 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.950120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.952208 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:02.952218 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.954894 (+ 2676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.954904 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:02.955190 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.955239 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 19:42:02.955350 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 19:42:02.955492 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1745,"mutex_wait_us":37,"spinlock_wait_cycles":31488,"thread_start_us":1174,"threads_started":1}
I20260806 19:42:02.956243 23951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:02.945041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:47133 (local address 127.21.202.66:32883)
0806 19:42:02.945303 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:02.945313 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:02.946811 (+ 1498us) server_negotiation.cc:407] Connection header received
0806 19:42:02.946955 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:02.946960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:02.947034 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:02.947130 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:02.948006 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.949969 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.952454 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:02.954638 (+ 2184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:02.954857 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:02.954938 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:02.955792 (+ 854us) server_negotiation.cc:299] Negotiation successful
0806 19:42:02.955944 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 19:42:02.957242 23748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3835c4caee734c068f54973dc166599d" candidate_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb861df2948c4ba3a936882765a971da" is_pre_election: true
I20260806 19:42:02.957063 23889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3835c4caee734c068f54973dc166599d" candidate_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18aace29b43a490e8c2b92d457c2eaa9" is_pre_election: true
I20260806 19:42:02.957796 23748 raft_consensus.cc:2454] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5b742f4edd0459ca802bcc95f1bb2ea in term 0.
I20260806 19:42:02.958073 23889 raft_consensus.cc:2454] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5b742f4edd0459ca802bcc95f1bb2ea in term 0.
I20260806 19:42:02.958547 23532 leader_election.cc:302] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5b742f4edd0459ca802bcc95f1bb2ea, eb861df2948c4ba3a936882765a971da; no voters:
I20260806 19:42:02.959064 23960 raft_consensus.cc:2781] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:42:02.959389 23960 raft_consensus.cc:482] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:02.959587 23960 raft_consensus.cc:3037] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:02.962426 23960 raft_consensus.cc:504] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } }
I20260806 19:42:02.963207 23960 leader_election.cc:288] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [CANDIDATE]: Term 1 election: Requested vote from peers 18aace29b43a490e8c2b92d457c2eaa9 (127.21.202.67:37343), eb861df2948c4ba3a936882765a971da (127.21.202.66:32883)
I20260806 19:42:02.963712 23889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3835c4caee734c068f54973dc166599d" candidate_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18aace29b43a490e8c2b92d457c2eaa9"
I20260806 19:42:02.963811 23748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3835c4caee734c068f54973dc166599d" candidate_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb861df2948c4ba3a936882765a971da"
I20260806 19:42:02.963967 23889 raft_consensus.cc:3037] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:02.964097 23748 raft_consensus.cc:3037] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:02.966975 23889 raft_consensus.cc:2454] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5b742f4edd0459ca802bcc95f1bb2ea in term 1.
I20260806 19:42:02.967011 23748 raft_consensus.cc:2454] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5b742f4edd0459ca802bcc95f1bb2ea in term 1.
I20260806 19:42:02.967478 23531 leader_election.cc:302] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18aace29b43a490e8c2b92d457c2eaa9, a5b742f4edd0459ca802bcc95f1bb2ea; no voters:
I20260806 19:42:02.967839 23960 raft_consensus.cc:2781] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:02.968461 23960 raft_consensus.cc:686] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [term 1 LEADER]: Becoming Leader. State: Replica: a5b742f4edd0459ca802bcc95f1bb2ea, State: Running, Role: LEADER
I20260806 19:42:02.969058 23960 consensus_queue.cc:237] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } }
W20260806 19:42:02.971879 23935 tablet.cc:2369] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:42:02.974270 23441 catalog_manager.cc:5495] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea reported cstate change: term changed from 0 to 1, leader changed from <none> to a5b742f4edd0459ca802bcc95f1bb2ea (127.21.202.65). New cstate: current_term: 1 leader_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } health_report { overall_health: HEALTHY } } }
I20260806 19:42:03.040046 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.034339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38634 (local address 127.21.202.65:36509)
0806 19:42:03.034602 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:03.034609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:03.034631 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:42:03.034995 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:03.034999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:03.035098 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:03.035242 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:03.035629 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.036324 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.037730 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.038137 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.038275 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.038768 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:03.039387 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:03.039550 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:03.039687 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 19:42:03.039769 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 19:42:03.050148 23748 raft_consensus.cc:1261] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da [term 1 FOLLOWER]: Refusing update from remote peer a5b742f4edd0459ca802bcc95f1bb2ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:03.051574 23964 consensus_queue.cc:1035] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:03.051870 23889 raft_consensus.cc:1261] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9 [term 1 FOLLOWER]: Refusing update from remote peer a5b742f4edd0459ca802bcc95f1bb2ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:03.052742 23964 consensus_queue.cc:1035] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:03.066774 23969 mvcc.cc:205] Tried to move back new op lower bound from 7315641643199778816 to 7315641642885251072. Current Snapshot: MvccSnapshot[applied={T|T < 7315641643199778816}]
I20260806 19:42:03.089934 23442 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:03.090345 23442 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:03.095188 23442 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e: Sending DeleteTablet for 3 replicas of tablet 3835c4caee734c068f54973dc166599d
I20260806 19:42:03.096755 23723 tablet_service.cc:1514] Processing DeleteTablet for tablet 3835c4caee734c068f54973dc166599d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:03 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:50350
I20260806 19:42:03.096761 23583 tablet_service.cc:1514] Processing DeleteTablet for tablet 3835c4caee734c068f54973dc166599d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:03 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:38626
I20260806 19:42:03.098230 23977 tablet_replica.cc:330] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da: stopping tablet replica
I20260806 19:42:03.099123 23977 raft_consensus.cc:2229] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:03.099442 23864 tablet_service.cc:1514] Processing DeleteTablet for tablet 3835c4caee734c068f54973dc166599d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:03 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:41442
I20260806 19:42:03.099866 23977 raft_consensus.cc:2258] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:03.100512 23979 tablet_replica.cc:330] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9: stopping tablet replica
I20260806 19:42:03.100773 23978 tablet_replica.cc:330] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea: stopping tablet replica
I20260806 19:42:03.101207 23979 raft_consensus.cc:2229] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:03.101249 23978 raft_consensus.cc:2229] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:42:03.101846 23979 raft_consensus.cc:2258] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:03.102135 23978 raft_consensus.cc:2258] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:03.102774 23441 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:42:03.103605 23441 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:42:03.104686 23977 ts_tablet_manager.cc:1905] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:03.105003 23979 ts_tablet_manager.cc:1905] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:03.105060 23978 ts_tablet_manager.cc:1905] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:03.112147 23979 ts_tablet_manager.cc:1918] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:03.112505 23979 log.cc:1196] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/3835c4caee734c068f54973dc166599d
I20260806 19:42:03.112556 23977 ts_tablet_manager.cc:1918] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:03.112739 23977 log.cc:1196] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/3835c4caee734c068f54973dc166599d
I20260806 19:42:03.113377 23977 ts_tablet_manager.cc:1939] T 3835c4caee734c068f54973dc166599d P eb861df2948c4ba3a936882765a971da: Deleting consensus metadata
I20260806 19:42:03.113430 23979 ts_tablet_manager.cc:1939] T 3835c4caee734c068f54973dc166599d P 18aace29b43a490e8c2b92d457c2eaa9: Deleting consensus metadata
I20260806 19:42:03.114003 23723 tablet_service.cc:1467] Processing CreateTablet for tablet 020275332785456799df4c6f16f8d785 (DEFAULT_TABLE table=test-table [id=1aab310fa34a40519c53fb63246334a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:03.114487 23723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 020275332785456799df4c6f16f8d785. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:03.115314 23978 ts_tablet_manager.cc:1918] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:03.115500 23978 log.cc:1196] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/3835c4caee734c068f54973dc166599d
I20260806 19:42:03.116080 23978 ts_tablet_manager.cc:1939] T 3835c4caee734c068f54973dc166599d P a5b742f4edd0459ca802bcc95f1bb2ea: Deleting consensus metadata
I20260806 19:42:03.117197 23427 catalog_manager.cc:4841] TS 18aace29b43a490e8c2b92d457c2eaa9 (127.21.202.67:37343): tablet 3835c4caee734c068f54973dc166599d (table test-table [id=7c5e275a71e7425099872993af77965a]) successfully deleted
I20260806 19:42:03.117322 23428 catalog_manager.cc:4841] TS eb861df2948c4ba3a936882765a971da (127.21.202.66:32883): tablet 3835c4caee734c068f54973dc166599d (table test-table [id=7c5e275a71e7425099872993af77965a]) successfully deleted
I20260806 19:42:03.117772 23429 catalog_manager.cc:4841] TS a5b742f4edd0459ca802bcc95f1bb2ea (127.21.202.65:36509): tablet 3835c4caee734c068f54973dc166599d (table test-table [id=7c5e275a71e7425099872993af77965a]) successfully deleted
I20260806 19:42:03.118209 23583 tablet_service.cc:1467] Processing CreateTablet for tablet 020275332785456799df4c6f16f8d785 (DEFAULT_TABLE table=test-table [id=1aab310fa34a40519c53fb63246334a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:03.118647 23583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 020275332785456799df4c6f16f8d785. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:03.113461 23864 tablet_service.cc:1467] Processing CreateTablet for tablet 020275332785456799df4c6f16f8d785 (DEFAULT_TABLE table=test-table [id=1aab310fa34a40519c53fb63246334a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:03.121536 23864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 020275332785456799df4c6f16f8d785. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:03.121706 23954 tablet_bootstrap.cc:492] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da: Bootstrap starting.
I20260806 19:42:03.123724 23954 tablet_bootstrap.cc:654] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:03.128551 23955 tablet_bootstrap.cc:492] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea: Bootstrap starting.
I20260806 19:42:03.129386 23953 tablet_bootstrap.cc:492] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9: Bootstrap starting.
I20260806 19:42:03.130322 23955 tablet_bootstrap.cc:654] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:03.131053 23953 tablet_bootstrap.cc:654] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:03.132393 23954 tablet_bootstrap.cc:492] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da: No bootstrap required, opened a new log
I20260806 19:42:03.132565 23954 ts_tablet_manager.cc:1397] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.001s
I20260806 19:42:03.133508 23954 raft_consensus.cc:348] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.133800 23954 raft_consensus.cc:374] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:03.133922 23954 raft_consensus.cc:729] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb861df2948c4ba3a936882765a971da, State: Initialized, Role: FOLLOWER
I20260806 19:42:03.134232 23954 consensus_queue.cc:260] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.138336 23954 ts_tablet_manager.cc:1428] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260806 19:42:03.141726 23953 tablet_bootstrap.cc:492] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9: No bootstrap required, opened a new log
I20260806 19:42:03.141719 23955 tablet_bootstrap.cc:492] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea: No bootstrap required, opened a new log
I20260806 19:42:03.141916 23953 ts_tablet_manager.cc:1397] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20260806 19:42:03.141942 23955 ts_tablet_manager.cc:1397] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20260806 19:42:03.142815 23953 raft_consensus.cc:348] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.142831 23955 raft_consensus.cc:348] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.143060 23953 raft_consensus.cc:374] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:03.143075 23955 raft_consensus.cc:374] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:03.143173 23953 raft_consensus.cc:729] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18aace29b43a490e8c2b92d457c2eaa9, State: Initialized, Role: FOLLOWER
I20260806 19:42:03.143178 23955 raft_consensus.cc:729] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5b742f4edd0459ca802bcc95f1bb2ea, State: Initialized, Role: FOLLOWER
I20260806 19:42:03.143458 23955 consensus_queue.cc:260] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.143464 23953 consensus_queue.cc:260] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.144114 23955 ts_tablet_manager.cc:1428] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 19:42:03.144692 23953 ts_tablet_manager.cc:1428] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
W20260806 19:42:03.361362 23654 tablet.cc:2369] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:42:03.454700 23959 raft_consensus.cc:482] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:03.454981 23959 raft_consensus.cc:504] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.456974 23959 leader_election.cc:288] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5b742f4edd0459ca802bcc95f1bb2ea (127.21.202.65:36509), eb861df2948c4ba3a936882765a971da (127.21.202.66:32883)
I20260806 19:42:03.465238 23985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.457566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:41609 (local address 127.21.202.66:32883)
0806 19:42:03.458547 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:03.458556 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:03.459105 (+ 549us) server_negotiation.cc:407] Connection header received
0806 19:42:03.459305 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:03.459327 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:03.459401 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:03.459511 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:03.460008 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.461056 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.462855 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.464115 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.464315 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.464404 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:03.464730 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 19:42:03.464903 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":1156,"threads_started":1}
I20260806 19:42:03.465267 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.457309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:32883 (local address 127.21.202.67:41609)
0806 19:42:03.459004 (+ 1695us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:03.459019 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:03.459160 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:03.459540 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:03.459546 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:03.459573 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:03.459839 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:03.459849 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.461204 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.461209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:03.462679 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:03.462693 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.464309 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.464315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:03.464514 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.464554 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 19:42:03.464675 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 19:42:03.464871 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1591,"thread_start_us":1397,"threads_started":1}
I20260806 19:42:03.466081 23748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020275332785456799df4c6f16f8d785" candidate_uuid: "18aace29b43a490e8c2b92d457c2eaa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb861df2948c4ba3a936882765a971da" is_pre_election: true
I20260806 19:42:03.466379 23748 raft_consensus.cc:2454] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18aace29b43a490e8c2b92d457c2eaa9 in term 0.
I20260806 19:42:03.466436 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.457916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:40591 (local address 127.21.202.65:36509)
0806 19:42:03.458113 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:03.458119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:03.458144 (+ 25us) server_negotiation.cc:407] Connection header received
0806 19:42:03.458235 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:03.458245 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:03.458329 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:03.458452 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:03.459375 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.460576 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.463208 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.464564 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.464752 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.464855 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:03.465990 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0806 19:42:03.466148 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 19:42:03.466835 23983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.457145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:36509 (local address 127.21.202.67:40591)
0806 19:42:03.457809 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:03.457846 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:03.458027 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:03.458873 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:03.458880 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:03.458913 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:03.459149 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:03.459162 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.460743 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.460749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:03.463030 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:03.463046 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.465079 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.465085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:03.465289 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.465782 (+ 493us) client_negotiation.cc:769] Sending connection context
0806 19:42:03.465937 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 19:42:03.466161 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":1402,"threads_started":1}
I20260806 19:42:03.466993 23814 leader_election.cc:302] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18aace29b43a490e8c2b92d457c2eaa9, eb861df2948c4ba3a936882765a971da; no voters:
I20260806 19:42:03.467489 23959 raft_consensus.cc:2781] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:42:03.467626 23959 raft_consensus.cc:482] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:03.467751 23959 raft_consensus.cc:3037] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:03.467854 23606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020275332785456799df4c6f16f8d785" candidate_uuid: "18aace29b43a490e8c2b92d457c2eaa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" is_pre_election: true
I20260806 19:42:03.468279 23606 raft_consensus.cc:2454] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18aace29b43a490e8c2b92d457c2eaa9 in term 0.
I20260806 19:42:03.471269 23959 raft_consensus.cc:504] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.472260 23959 leader_election.cc:288] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [CANDIDATE]: Term 1 election: Requested vote from peers a5b742f4edd0459ca802bcc95f1bb2ea (127.21.202.65:36509), eb861df2948c4ba3a936882765a971da (127.21.202.66:32883)
I20260806 19:42:03.472901 23606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020275332785456799df4c6f16f8d785" candidate_uuid: "18aace29b43a490e8c2b92d457c2eaa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea"
I20260806 19:42:03.472891 23748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020275332785456799df4c6f16f8d785" candidate_uuid: "18aace29b43a490e8c2b92d457c2eaa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb861df2948c4ba3a936882765a971da"
I20260806 19:42:03.473143 23748 raft_consensus.cc:3037] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:03.473170 23606 raft_consensus.cc:3037] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:03.475557 23606 raft_consensus.cc:2454] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18aace29b43a490e8c2b92d457c2eaa9 in term 1.
I20260806 19:42:03.476046 23815 leader_election.cc:302] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18aace29b43a490e8c2b92d457c2eaa9, a5b742f4edd0459ca802bcc95f1bb2ea; no voters:
I20260806 19:42:03.476346 23748 raft_consensus.cc:2454] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18aace29b43a490e8c2b92d457c2eaa9 in term 1.
I20260806 19:42:03.476426 23959 raft_consensus.cc:2781] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:03.477077 23959 raft_consensus.cc:686] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [term 1 LEADER]: Becoming Leader. State: Replica: 18aace29b43a490e8c2b92d457c2eaa9, State: Running, Role: LEADER
I20260806 19:42:03.477550 23959 consensus_queue.cc:237] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.481730 23441 catalog_manager.cc:5495] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18aace29b43a490e8c2b92d457c2eaa9 (127.21.202.67). New cstate: current_term: 1 leader_uuid: "18aace29b43a490e8c2b92d457c2eaa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } health_report { overall_health: HEALTHY } } }
Aug 06 19:42:03 dist-test-slave-77v9 krb5kdc[23397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045320, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:03.535472 23993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.519423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:43777 (local address 127.0.0.1:33872)
0806 19:42:03.520143 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:03.520172 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:03.520360 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:03.520747 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:03.520754 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:03.521810 (+ 1056us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:03.522120 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:03.522133 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.523558 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.523568 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:03.524912 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:03.524932 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.525765 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.525772 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:03.525962 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.526725 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:03.526744 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:03.529330 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:03.532854 (+ 3524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:03.532871 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:03.532879 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:03.533315 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:03.533680 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:03.533687 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:03.533690 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:03.533848 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:03.534378 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:03.534386 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:03.534543 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 19:42:03.534717 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 19:42:03.535011 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":355,"threads_started":1}
I20260806 19:42:03.535490 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.519474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33872 (local address 127.21.202.126:43777)
0806 19:42:03.520490 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:03.520494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:03.520512 (+ 18us) server_negotiation.cc:407] Connection header received
0806 19:42:03.520551 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:03.520554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:03.520605 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:03.520714 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:03.522300 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.523353 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.525070 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.525559 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.525737 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.529585 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:03.529590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:03.529593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:03.529625 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:03.532679 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:03.533448 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:03.533457 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:03.533459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:03.533575 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:03.533957 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:03.533962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:03.533965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:03.534233 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:03.534394 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:03.534809 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 19:42:03.535134 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":879,"thread_start_us":718,"threads_started":1}
I20260806 19:42:03.541800 23441 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:33872:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 19:42:03.543010 23441 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:42:03.550086 23583 tablet_service.cc:1467] Processing CreateTablet for tablet 2a0c4a81600b415ca43361a30d8bb66b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7170ab2fa58d4bcebd98ed13dafa5c6a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:42:03.550395 23723 tablet_service.cc:1467] Processing CreateTablet for tablet 2a0c4a81600b415ca43361a30d8bb66b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7170ab2fa58d4bcebd98ed13dafa5c6a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:42:03.550729 23583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a0c4a81600b415ca43361a30d8bb66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:03.551023 23723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a0c4a81600b415ca43361a30d8bb66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:03.550886 23864 tablet_service.cc:1467] Processing CreateTablet for tablet 2a0c4a81600b415ca43361a30d8bb66b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7170ab2fa58d4bcebd98ed13dafa5c6a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:42:03.551501 23864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a0c4a81600b415ca43361a30d8bb66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:03.556689 23955 tablet_bootstrap.cc:492] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea: Bootstrap starting.
I20260806 19:42:03.558045 23955 tablet_bootstrap.cc:654] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:03.559170 23954 tablet_bootstrap.cc:492] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da: Bootstrap starting.
I20260806 19:42:03.559736 23953 tablet_bootstrap.cc:492] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9: Bootstrap starting.
I20260806 19:42:03.560935 23955 tablet_bootstrap.cc:492] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea: No bootstrap required, opened a new log
I20260806 19:42:03.561003 23954 tablet_bootstrap.cc:654] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:03.561107 23955 ts_tablet_manager.cc:1397] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20260806 19:42:03.561604 23953 tablet_bootstrap.cc:654] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:03.561904 23955 raft_consensus.cc:348] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.562253 23955 raft_consensus.cc:374] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:03.562361 23955 raft_consensus.cc:729] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5b742f4edd0459ca802bcc95f1bb2ea, State: Initialized, Role: FOLLOWER
I20260806 19:42:03.562646 23955 consensus_queue.cc:260] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.563316 23972 tablet_replica.cc:439] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } }
I20260806 19:42:03.563719 23972 tablet_replica.cc:442] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:03.563755 23954 tablet_bootstrap.cc:492] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da: No bootstrap required, opened a new log
I20260806 19:42:03.563930 23954 ts_tablet_manager.cc:1397] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 19:42:03.564034 23955 ts_tablet_manager.cc:1428] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 19:42:03.564884 23953 tablet_bootstrap.cc:492] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9: No bootstrap required, opened a new log
I20260806 19:42:03.564747 23954 raft_consensus.cc:348] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.565078 23953 ts_tablet_manager.cc:1397] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 19:42:03.565079 23954 raft_consensus.cc:374] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:03.565217 23954 raft_consensus.cc:729] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb861df2948c4ba3a936882765a971da, State: Initialized, Role: FOLLOWER
I20260806 19:42:03.565550 23954 consensus_queue.cc:260] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.566030 23953 raft_consensus.cc:348] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.566237 23954 ts_tablet_manager.cc:1428] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 19:42:03.566272 23953 raft_consensus.cc:374] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:03.566358 23953 raft_consensus.cc:729] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18aace29b43a490e8c2b92d457c2eaa9, State: Initialized, Role: FOLLOWER
I20260806 19:42:03.566576 23961 tablet_replica.cc:439] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } }
I20260806 19:42:03.566648 23953 consensus_queue.cc:260] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.566931 23961 tablet_replica.cc:442] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:03.567266 23953 ts_tablet_manager.cc:1428] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 19:42:03.567143 23959 tablet_replica.cc:439] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } }
I20260806 19:42:03.567500 23959 tablet_replica.cc:442] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:03.589951 23972 raft_consensus.cc:482] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:03.590166 23972 raft_consensus.cc:504] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.590875 23972 leader_election.cc:288] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb861df2948c4ba3a936882765a971da (127.21.202.66:32883), 18aace29b43a490e8c2b92d457c2eaa9 (127.21.202.67:37343)
I20260806 19:42:03.591398 23748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0c4a81600b415ca43361a30d8bb66b" candidate_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb861df2948c4ba3a936882765a971da" is_pre_election: true
I20260806 19:42:03.591426 23889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0c4a81600b415ca43361a30d8bb66b" candidate_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18aace29b43a490e8c2b92d457c2eaa9" is_pre_election: true
I20260806 19:42:03.591674 23748 raft_consensus.cc:2454] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5b742f4edd0459ca802bcc95f1bb2ea in term 0.
I20260806 19:42:03.591668 23889 raft_consensus.cc:2454] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5b742f4edd0459ca802bcc95f1bb2ea in term 0.
I20260806 19:42:03.592128 23532 leader_election.cc:302] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5b742f4edd0459ca802bcc95f1bb2ea, eb861df2948c4ba3a936882765a971da; no voters:
I20260806 19:42:03.592414 23972 raft_consensus.cc:2781] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:42:03.592545 23972 raft_consensus.cc:482] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:03.592650 23972 raft_consensus.cc:3037] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:03.594816 23972 raft_consensus.cc:504] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.595348 23972 leader_election.cc:288] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [CANDIDATE]: Term 1 election: Requested vote from peers eb861df2948c4ba3a936882765a971da (127.21.202.66:32883), 18aace29b43a490e8c2b92d457c2eaa9 (127.21.202.67:37343)
I20260806 19:42:03.595798 23748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0c4a81600b415ca43361a30d8bb66b" candidate_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb861df2948c4ba3a936882765a971da"
I20260806 19:42:03.595818 23889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0c4a81600b415ca43361a30d8bb66b" candidate_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18aace29b43a490e8c2b92d457c2eaa9"
I20260806 19:42:03.596014 23889 raft_consensus.cc:3037] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:03.596056 23748 raft_consensus.cc:3037] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:03.599057 23889 raft_consensus.cc:2454] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5b742f4edd0459ca802bcc95f1bb2ea in term 1.
I20260806 19:42:03.599434 23748 raft_consensus.cc:2454] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5b742f4edd0459ca802bcc95f1bb2ea in term 1.
I20260806 19:42:03.599481 23531 leader_election.cc:302] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18aace29b43a490e8c2b92d457c2eaa9, a5b742f4edd0459ca802bcc95f1bb2ea; no voters:
I20260806 19:42:03.599822 23972 raft_consensus.cc:2781] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:03.600255 23972 raft_consensus.cc:686] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [term 1 LEADER]: Becoming Leader. State: Replica: a5b742f4edd0459ca802bcc95f1bb2ea, State: Running, Role: LEADER
I20260806 19:42:03.600561 23972 consensus_queue.cc:237] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } }
I20260806 19:42:03.602733 23999 tablet_replica.cc:439] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea: TxnStatusTablet state changed. Reason: New leader a5b742f4edd0459ca802bcc95f1bb2ea. Latest consensus state: current_term: 1 leader_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } }
I20260806 19:42:03.603116 23999 tablet_replica.cc:442] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:42:03.604351 23442 catalog_manager.cc:5495] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea reported cstate change: term changed from 0 to 1, leader changed from <none> to a5b742f4edd0459ca802bcc95f1bb2ea (127.21.202.65). New cstate: current_term: 1 leader_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } health_report { overall_health: UNKNOWN } } }
I20260806 19:42:03.604593 24001 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 19:42:03.604971 24001 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 19:42:03.634224 23993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.626921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:36509 (local address 127.0.0.1:38642)
0806 19:42:03.627207 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:03.627227 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:03.627361 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:03.627817 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:03.627822 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:03.627837 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:42:03.628058 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:03.628069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.629682 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.629688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:03.631560 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:03.631577 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.632828 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.632835 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:03.633054 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.633143 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:42:03.633633 (+ 490us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:42:03.633667 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 19:42:03.633768 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 19:42:03.633968 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 19:42:03.634248 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.626896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38642 (local address 127.21.202.65:36509)
0806 19:42:03.627352 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:03.627359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:03.627381 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:42:03.627551 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:03.627557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:03.627643 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:03.627760 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:03.628237 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.629527 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.631742 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.632616 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.632767 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.633357 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:03.633513 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:03.633620 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:03.633808 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 19:42:03.633900 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232}
I20260806 19:42:03.639525 23889 raft_consensus.cc:1261] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9 [term 1 FOLLOWER]: Refusing update from remote peer a5b742f4edd0459ca802bcc95f1bb2ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:03.639613 23748 raft_consensus.cc:1261] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da [term 1 FOLLOWER]: Refusing update from remote peer a5b742f4edd0459ca802bcc95f1bb2ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:03.640523 23999 consensus_queue.cc:1035] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:03.640945 23972 consensus_queue.cc:1035] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:03.645167 23959 tablet_replica.cc:439] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9: TxnStatusTablet state changed. Reason: New leader a5b742f4edd0459ca802bcc95f1bb2ea. Latest consensus state: current_term: 1 leader_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } }
I20260806 19:42:03.645485 23959 tablet_replica.cc:442] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:03.646809 24003 mvcc.cc:205] Tried to move back new op lower bound from 7315641645620121600 to 7315641645471084544. Current Snapshot: MvccSnapshot[applied={T|T < 7315641645620121600}]
I20260806 19:42:03.653316 23972 tablet_replica.cc:439] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } }
I20260806 19:42:03.653647 23972 tablet_replica.cc:442] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:42:03.664563 23959 tablet_replica.cc:439] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } }
I20260806 19:42:03.664958 23959 tablet_replica.cc:442] T 2a0c4a81600b415ca43361a30d8bb66b P 18aace29b43a490e8c2b92d457c2eaa9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:03.671058 23961 tablet_replica.cc:439] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da: TxnStatusTablet state changed. Reason: New leader a5b742f4edd0459ca802bcc95f1bb2ea. Latest consensus state: current_term: 1 leader_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } }
I20260806 19:42:03.671404 23961 tablet_replica.cc:442] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:03.672363 23999 tablet_replica.cc:439] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } }
I20260806 19:42:03.672674 23999 tablet_replica.cc:442] T 2a0c4a81600b415ca43361a30d8bb66b P a5b742f4edd0459ca802bcc95f1bb2ea: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:42:03.677644 23961 tablet_replica.cc:439] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 } } peers { permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 } } peers { permanent_uuid: "18aace29b43a490e8c2b92d457c2eaa9" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 37343 } } }
I20260806 19:42:03.677898 23961 tablet_replica.cc:442] T 2a0c4a81600b415ca43361a30d8bb66b P eb861df2948c4ba3a936882765a971da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:03.687276 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.672448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41456 (local address 127.21.202.67:37343)
0806 19:42:03.677100 (+ 4652us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:03.677110 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:03.680911 (+ 3801us) server_negotiation.cc:407] Connection header received
0806 19:42:03.681235 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:03.681242 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:03.681382 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:03.681583 (+ 201us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:03.681987 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.683159 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.684746 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.685252 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.685417 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.685891 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:03.686583 (+ 692us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:03.686743 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:03.686833 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 19:42:03.686921 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4457,"thread_start_us":4242,"threads_started":1}
I20260806 19:42:03.705974 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.699093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:36789 (local address 127.21.202.65:36509)
0806 19:42:03.699342 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:03.699352 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:03.699981 (+ 629us) server_negotiation.cc:407] Connection header received
0806 19:42:03.700158 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:03.700163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:03.700242 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:03.700334 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:03.701016 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.701921 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.704042 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.704548 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.704655 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.705240 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:03.705376 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:03.705456 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:03.705635 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 19:42:03.705706 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 19:42:03.706250 24016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.699101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:36509 (local address 127.21.202.67:36789)
0806 19:42:03.699849 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:03.699870 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:03.700029 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:03.700375 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:03.700380 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:03.700393 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:42:03.700797 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:03.700836 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.702062 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.702068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:03.703861 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:03.703878 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.704738 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.704745 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:03.704962 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.705036 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:42:03.705482 (+ 446us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:42:03.705494 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 19:42:03.705583 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 19:42:03.705730 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":388,"threads_started":1}
I20260806 19:42:03.717604 23748 raft_consensus.cc:1261] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da [term 1 FOLLOWER]: Refusing update from remote peer 18aace29b43a490e8c2b92d457c2eaa9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:03.717890 23606 raft_consensus.cc:1261] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea [term 1 FOLLOWER]: Refusing update from remote peer 18aace29b43a490e8c2b92d457c2eaa9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:03.718621 23959 consensus_queue.cc:1035] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb861df2948c4ba3a936882765a971da" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 32883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:03.719048 23986 consensus_queue.cc:1035] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5b742f4edd0459ca802bcc95f1bb2ea" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:03.783886 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.775872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:32927 (local address 127.21.202.67:37343)
0806 19:42:03.776051 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:03.776059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:03.777244 (+ 1185us) server_negotiation.cc:407] Connection header received
0806 19:42:03.777512 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:03.777519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:03.777618 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:03.777730 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:03.778372 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.779525 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.781723 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.782409 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.782528 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.783111 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:03.783245 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:03.783341 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:03.783478 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 19:42:03.783567 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 19:42:03.784044 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.776244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:37343 (local address 127.21.202.65:32927)
0806 19:42:03.777109 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:03.777131 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:03.777303 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:03.777796 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:03.777801 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:03.777813 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:42:03.778199 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:03.778209 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.779652 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.779657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:03.781570 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:03.781585 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.782633 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.782641 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:03.782807 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.782873 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:42:03.783352 (+ 479us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:42:03.783362 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 19:42:03.783438 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 19:42:03.783564 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":451,"threads_started":1}
I20260806 19:42:03.835114 23442 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:03.835417 23442 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:03.840373 23442 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f78567e110ef4edfa792340b7921161e: Sending DeleteTablet for 3 replicas of tablet 020275332785456799df4c6f16f8d785
I20260806 19:42:03.841465 23723 tablet_service.cc:1514] Processing DeleteTablet for tablet 020275332785456799df4c6f16f8d785 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:03 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:50350
I20260806 19:42:03.841557 23864 tablet_service.cc:1514] Processing DeleteTablet for tablet 020275332785456799df4c6f16f8d785 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:03 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:41442
I20260806 19:42:03.842556 23583 tablet_service.cc:1514] Processing DeleteTablet for tablet 020275332785456799df4c6f16f8d785 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:03 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:38626
I20260806 19:42:03.857465 24026 tablet_replica.cc:330] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9: stopping tablet replica
I20260806 19:42:03.857733 24026 raft_consensus.cc:2229] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:42:03.858448 24026 raft_consensus.cc:2258] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:03.859802 24026 ts_tablet_manager.cc:1905] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:03.869005 24026 ts_tablet_manager.cc:1918] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:42:03.869172 24026 log.cc:1196] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/020275332785456799df4c6f16f8d785
I20260806 19:42:03.869663 24026 ts_tablet_manager.cc:1939] T 020275332785456799df4c6f16f8d785 P 18aace29b43a490e8c2b92d457c2eaa9: Deleting consensus metadata
I20260806 19:42:03.871165 23427 catalog_manager.cc:4841] TS 18aace29b43a490e8c2b92d457c2eaa9 (127.21.202.67:37343): tablet 020275332785456799df4c6f16f8d785 (table test-table [id=1aab310fa34a40519c53fb63246334a8]) successfully deleted
I20260806 19:42:03.874625 24025 tablet_replica.cc:330] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da: stopping tablet replica
I20260806 19:42:03.874722 24027 tablet_replica.cc:330] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea: stopping tablet replica
I20260806 19:42:03.875007 24027 raft_consensus.cc:2229] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:03.875010 24025 raft_consensus.cc:2229] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:03.875420 24025 raft_consensus.cc:2258] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:03.875468 24027 raft_consensus.cc:2258] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:03.877101 24025 ts_tablet_manager.cc:1905] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:03.878384 24027 ts_tablet_manager.cc:1905] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:03.885762 24027 ts_tablet_manager.cc:1918] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:42:03.885939 24027 log.cc:1196] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/020275332785456799df4c6f16f8d785
I20260806 19:42:03.886314 24025 ts_tablet_manager.cc:1918] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:42:03.886437 24027 ts_tablet_manager.cc:1939] T 020275332785456799df4c6f16f8d785 P a5b742f4edd0459ca802bcc95f1bb2ea: Deleting consensus metadata
I20260806 19:42:03.886600 24025 log.cc:1196] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/020275332785456799df4c6f16f8d785
I20260806 19:42:03.887301 24025 ts_tablet_manager.cc:1939] T 020275332785456799df4c6f16f8d785 P eb861df2948c4ba3a936882765a971da: Deleting consensus metadata
I20260806 19:42:03.887585 23429 catalog_manager.cc:4841] TS a5b742f4edd0459ca802bcc95f1bb2ea (127.21.202.65:36509): tablet 020275332785456799df4c6f16f8d785 (table test-table [id=1aab310fa34a40519c53fb63246334a8]) successfully deleted
I20260806 19:42:03.888958 23428 catalog_manager.cc:4841] TS eb861df2948c4ba3a936882765a971da (127.21.202.66:32883): tablet 020275332785456799df4c6f16f8d785 (table test-table [id=1aab310fa34a40519c53fb63246334a8]) successfully deleted
Aug 06 19:42:03 dist-test-slave-77v9 krb5kdc[23397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045322, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:42:03.907341 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.863073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38646 (local address 127.21.202.65:36509)
0806 19:42:03.863313 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:03.863324 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:03.876731 (+ 13407us) server_negotiation.cc:407] Connection header received
0806 19:42:03.877381 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:03.877387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:03.877469 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:03.877553 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:03.879951 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.881158 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.883289 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.884007 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.884118 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.898387 (+ 14269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:03.898412 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:03.898442 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:03.898513 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:03.903460 (+ 4947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:03.904488 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:03.904500 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:03.904505 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:03.904704 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:03.905209 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:03.905215 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:03.905220 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:03.905773 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:03.906145 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:03.906571 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 19:42:03.906847 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260806 19:42:03.908398 23648 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38646
I20260806 19:42:03.939252 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.913000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33884 (local address 127.21.202.126:43777)
0806 19:42:03.913235 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:03.913243 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:03.917401 (+ 4158us) server_negotiation.cc:407] Connection header received
0806 19:42:03.917629 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:03.917636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:03.917745 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:03.917863 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:03.919849 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.921252 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.923358 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.924228 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.924469 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.931153 (+ 6684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:03.931161 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:03.931166 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:03.931216 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:03.935503 (+ 4287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:03.936486 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:03.936492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:03.936496 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:03.936660 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:03.937197 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:03.937203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:03.937207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:03.937633 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:03.937885 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:03.938596 (+ 711us) server_negotiation.cc:299] Negotiation successful
0806 19:42:03.938867 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260806 19:42:03.940627 23442 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33884
I20260806 19:42:03.943427 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 23515
I20260806 19:42:03.977613 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:03.945467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38648 (local address 127.21.202.65:36509)
0806 19:42:03.945728 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:03.945736 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:03.957493 (+ 11757us) server_negotiation.cc:407] Connection header received
0806 19:42:03.957694 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:03.957701 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:03.957813 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:03.957956 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:03.960126 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.961756 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.964079 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:03.965028 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:03.965164 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:03.969249 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:03.969257 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:03.969263 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:03.969323 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:03.973855 (+ 4532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:03.974829 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:03.974835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:03.974839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:03.975023 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:03.975535 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:03.975540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:03.975544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:03.975926 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:03.976193 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:03.976911 (+ 718us) server_negotiation.cc:299] Negotiation successful
0806 19:42:03.977172 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260806 19:42:03.978988 23648 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38648
W20260806 19:42:03.979694 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 23515: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:42:03.979874 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 23515
I20260806 19:42:04.009745 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 23657
Aug 06 19:42:04 dist-test-slave-77v9 krb5kdc[23397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045322, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:42:04.030901 24033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:04.010665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50352 (local address 127.21.202.66:32883)
0806 19:42:04.014458 (+ 3793us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:04.014463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:04.014481 (+ 18us) server_negotiation.cc:407] Connection header received
0806 19:42:04.014540 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:04.014544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:04.014620 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:04.014753 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:04.016078 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:04.016980 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:04.018409 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:04.018870 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:04.019025 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:04.024032 (+ 5007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:04.024047 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:04.024065 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:04.024103 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:04.027435 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:04.028207 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:04.028216 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:04.028218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:04.028360 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:04.028985 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:04.028990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:04.028993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:04.029492 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:04.029906 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:04.030189 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 19:42:04.030405 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3676,"thread_start_us":3539,"threads_started":1}
W20260806 19:42:04.031951 23788 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50352
W20260806 19:42:04.032765 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 23657: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:42:04.033012 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 23657
I20260806 19:42:04.059638 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 23797
Aug 06 19:42:04 dist-test-slave-77v9 krb5kdc[23397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045322, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:42:04.078711 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:04.060534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41466 (local address 127.21.202.67:37343)
0806 19:42:04.060765 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:04.060770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:04.060790 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:42:04.061055 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:04.061060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:04.061157 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:04.061260 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:04.062915 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:04.063819 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:04.065684 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:04.066158 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:04.066270 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:04.071891 (+ 5621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:04.071911 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:04.071930 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:04.071976 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:04.075416 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:04.076374 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:04.076382 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:04.076385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:04.076520 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:04.077002 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:04.077006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:04.077008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:04.077465 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:04.077788 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:04.078115 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 19:42:04.078310 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260806 19:42:04.079535 23929 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41466
W20260806 19:42:04.080294 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 23797: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:42:04.080493 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 23797
I20260806 19:42:04.101970 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 23410
I20260806 19:42:04.103336 23483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:04.228600 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 23410
2026-08-06T19:42:04Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5204 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:42:04 dist-test-slave-77v9 krb5kdc[24037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:42:04 dist-test-slave-77v9 krb5kdc[24037](info): set up 2 sockets
Aug 06 19:42:04 dist-test-slave-77v9 krb5kdc[24037](info): commencing operation
krb5kdc: starting...
W20260806 19:42:05.362434 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:42:05 dist-test-slave-77v9 krb5kdc[24037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:42:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:42:05.598042 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:38963
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:43883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:38963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:05.945760 24050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:05.946118 24050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:05.946281 24050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:05.953677 24050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:05.953828 24050 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 19:42:05.953923 24050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:05.954035 24050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:05.954118 24050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:05.954206 24050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:42:05.964510 24050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:43883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:38963
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:38963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:05.965951 24050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:05.967379 24050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:05.979405 24055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:05.979317 24056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:05.979423 24058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:05.980458 24050 server_base.cc:1034] running on GCE node
I20260806 19:42:05.981302 24050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:05.982755 24050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:05.984007 24050 hybrid_clock.cc:648] HybridClock initialized: now 1786045325983948 us; error 29 us; skew 500 ppm
I20260806 19:42:05.984390 24050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:05 dist-test-slave-77v9 krb5kdc[24037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045325, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:05.987691 24050 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:42:05.989234 24050 webserver.cc:460] Webserver started at http://127.21.202.126:38129/ using document root <none> and password file <none>
I20260806 19:42:05.989960 24050 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:05.990083 24050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:05.990401 24050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:05.993395 24050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "0439f0f563e3412f8173359deac61e6e"
format_stamp: "Formatted at 2026-08-06 19:42:05 on dist-test-slave-77v9"
server_key: "880ab57235a11c37fce202c7a84a0ae2"
server_key_iv: "ec88e81b7bdb4207385f95a0d6a8ac28"
server_key_version: "encryptionkey@0"
I20260806 19:42:05.994172 24050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "0439f0f563e3412f8173359deac61e6e"
format_stamp: "Formatted at 2026-08-06 19:42:05 on dist-test-slave-77v9"
server_key: "880ab57235a11c37fce202c7a84a0ae2"
server_key_iv: "ec88e81b7bdb4207385f95a0d6a8ac28"
server_key_version: "encryptionkey@0"
I20260806 19:42:05.999706 24050 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:42:06.003448 24065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:06.004884 24050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 19:42:06.005075 24050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "0439f0f563e3412f8173359deac61e6e"
format_stamp: "Formatted at 2026-08-06 19:42:05 on dist-test-slave-77v9"
server_key: "880ab57235a11c37fce202c7a84a0ae2"
server_key_iv: "ec88e81b7bdb4207385f95a0d6a8ac28"
server_key_version: "encryptionkey@0"
I20260806 19:42:06.005303 24050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:06.021088 24050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:06.024982 24050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:06.025260 24050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:06.047200 24050 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:38963
I20260806 19:42:06.047246 24128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:38963 every 8 connection(s)
I20260806 19:42:06.048718 24050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:06.051793 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 24050
I20260806 19:42:06.052026 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:06.052922 22313 external_mini_cluster.cc:1428] Setting key a2209f581f8b361dd6c828ed826020c8
I20260806 19:42:06.053435 24129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 19:42:06 dist-test-slave-77v9 krb5kdc[24037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:06.068529 24129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e: Bootstrap starting.
I20260806 19:42:06.071938 24129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:06.073107 24129 log.cc:824] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:06.076000 24129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e: No bootstrap required, opened a new log
I20260806 19:42:06.077108 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:06.055054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.21.202.126:38963)
0806 19:42:06.055823 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:06.055835 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:06.055874 (+ 39us) server_negotiation.cc:407] Connection header received
0806 19:42:06.058119 (+ 2245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:06.058141 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:06.058538 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:06.058854 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:06.060733 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:06.062080 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:06.063823 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:06.064542 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:06.064695 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:06.069596 (+ 4901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:06.069604 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:06.069619 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:06.069657 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:06.072768 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:06.073559 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:06.073573 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:06.073581 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:06.073758 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:06.074185 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:06.074190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:06.074194 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:06.074738 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:06.075037 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:06.075549 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 19:42:06.076003 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":374,"threads_started":1}
I20260806 19:42:06.085294 24129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0439f0f563e3412f8173359deac61e6e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38963 } }
I20260806 19:42:06.085732 24129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:06.085855 24129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0439f0f563e3412f8173359deac61e6e, State: Initialized, Role: FOLLOWER
I20260806 19:42:06.086463 24129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0439f0f563e3412f8173359deac61e6e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38963 } }
I20260806 19:42:06.086720 24129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:06.086880 24129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:06.087057 24129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:06.089758 24129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0439f0f563e3412f8173359deac61e6e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38963 } }
I20260806 19:42:06.090324 24129 leader_election.cc:302] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0439f0f563e3412f8173359deac61e6e; no voters:
I20260806 19:42:06.090962 24129 leader_election.cc:288] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:06.091135 24140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:06.091888 24140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [term 1 LEADER]: Becoming Leader. State: Replica: 0439f0f563e3412f8173359deac61e6e, State: Running, Role: LEADER
I20260806 19:42:06.092370 24140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0439f0f563e3412f8173359deac61e6e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38963 } }
I20260806 19:42:06.093344 24129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:06.095391 24141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0439f0f563e3412f8173359deac61e6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0439f0f563e3412f8173359deac61e6e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38963 } } }
I20260806 19:42:06.095784 24141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:06.098933 24142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0439f0f563e3412f8173359deac61e6e. Latest consensus state: current_term: 1 leader_uuid: "0439f0f563e3412f8173359deac61e6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0439f0f563e3412f8173359deac61e6e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38963 } } }
I20260806 19:42:06.099231 24142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:06.100545 24147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:06.105562 24147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:06.115188 24147 catalog_manager.cc:1347] Generated new cluster ID: 6e5fd28f86544478853e1771da13e007
I20260806 19:42:06.115339 24147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:06.126031 24147 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:06.127358 24147 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:06.139195 24147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e: Generated new TSK 0
I20260806 19:42:06.140094 24147 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:42:06.202764 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:38963
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:43883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:06.564738 24161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:06.565187 24161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:06.565352 24161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:06.573076 24161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:06.573233 24161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:06.573382 24161 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 19:42:06.573472 24161 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:42:06.573599 24161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:42:06.584862 24161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:43883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.202.126:38963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:06.586359 24161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:06.587734 24161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:06.602304 24169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:06.602356 24166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:06.602272 24167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:06.606997 24161 server_base.cc:1034] running on GCE node
I20260806 19:42:06.607764 24161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:06.609229 24161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:06.610584 24161 hybrid_clock.cc:648] HybridClock initialized: now 1786045326610454 us; error 112 us; skew 500 ppm
I20260806 19:42:06.611153 24161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:06 dist-test-slave-77v9 krb5kdc[24037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045326, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:06.615196 24161 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:42:06.616995 24161 webserver.cc:460] Webserver started at http://127.21.202.65:37091/ using document root <none> and password file <none>
I20260806 19:42:06.617738 24161 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:06.617846 24161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:06.618127 24161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:06.621384 24161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "8492820db869485582cc539ba164fac6"
format_stamp: "Formatted at 2026-08-06 19:42:06 on dist-test-slave-77v9"
server_key: "5b67acf585892a596fb7abcb17969cda"
server_key_iv: "8792467b30fedeb682a506eb5cfc8ced"
server_key_version: "encryptionkey@0"
I20260806 19:42:06.622221 24161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "8492820db869485582cc539ba164fac6"
format_stamp: "Formatted at 2026-08-06 19:42:06 on dist-test-slave-77v9"
server_key: "5b67acf585892a596fb7abcb17969cda"
server_key_iv: "8792467b30fedeb682a506eb5cfc8ced"
server_key_version: "encryptionkey@0"
I20260806 19:42:06.627954 24161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:42:06.631942 24176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:06.633507 24161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:06.633708 24161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "8492820db869485582cc539ba164fac6"
format_stamp: "Formatted at 2026-08-06 19:42:06 on dist-test-slave-77v9"
server_key: "5b67acf585892a596fb7abcb17969cda"
server_key_iv: "8792467b30fedeb682a506eb5cfc8ced"
server_key_version: "encryptionkey@0"
I20260806 19:42:06.633914 24161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:06.661689 24161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:06.666210 24161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:06.666520 24161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:06.669570 24161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:06.669708 24161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:06.669838 24161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:06.669931 24161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:42:06 dist-test-slave-77v9 krb5kdc[24037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045326, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:06.696529 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:06.670262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:43353 (local address 127.21.202.126:38963)
0806 19:42:06.673306 (+ 3044us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:06.673311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:06.673333 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:42:06.675872 (+ 2539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:06.675882 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:06.676005 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:06.676144 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:06.678830 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:06.679932 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:06.681741 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:06.682185 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:06.682355 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:06.687562 (+ 5207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:06.687571 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:06.687578 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:06.687639 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:06.691422 (+ 3783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:06.692732 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:06.692740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:06.692744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:06.692962 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:06.693813 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:06.693821 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:06.693827 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:06.694221 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:06.694452 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:06.695559 (+ 1107us) server_negotiation.cc:299] Negotiation successful
0806 19:42:06.695893 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2921,"thread_start_us":2797,"threads_started":1}
I20260806 19:42:06.697266 24186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:06.671109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:38963 (local address 127.21.202.65:43353)
0806 19:42:06.672233 (+ 1124us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:06.672300 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:06.675476 (+ 3176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:06.676542 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:06.676558 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:06.677551 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:06.678604 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:06.678641 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:06.680091 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:06.680099 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:06.681595 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:06.681607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:06.682510 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:06.682521 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:06.682942 (+ 421us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:06.683895 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:06.683919 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:06.687283 (+ 3364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:06.691614 (+ 4331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:06.691632 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:06.691658 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:06.692525 (+ 867us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:06.693134 (+ 609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:06.693145 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:06.693151 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:06.693556 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:06.694553 (+ 997us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:06.694569 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:06.695075 (+ 506us) client_negotiation.cc:769] Sending connection context
0806 19:42:06.695351 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 19:42:06.695782 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":464,"threads_started":1}
I20260806 19:42:06.719750 24161 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:37029
I20260806 19:42:06.719796 24298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:37029 every 8 connection(s)
I20260806 19:42:06.720803 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:06.703644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:42433 (local address 127.21.202.126:38963)
0806 19:42:06.703816 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:06.703820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:06.704521 (+ 701us) server_negotiation.cc:407] Connection header received
0806 19:42:06.704695 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:06.704703 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:06.704832 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:06.704953 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:06.706864 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:06.708116 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:06.710684 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:06.711244 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:06.711405 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:06.714316 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:06.714321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:06.714324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:06.714359 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:06.717873 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:06.718575 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:06.718581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:06.718585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:06.718729 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:06.719234 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:06.719237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:06.719239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:06.719525 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:06.719679 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:06.720207 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 19:42:06.720441 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 19:42:06.720954 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:06.703734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:38963 (local address 127.21.202.65:42433)
0806 19:42:06.704417 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:06.704435 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:06.704565 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:06.705157 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:06.705163 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:06.706056 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:06.706700 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:06.706716 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:06.708354 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:06.708363 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:06.710509 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:06.710532 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:06.711522 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:06.711531 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:06.711778 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:06.712687 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:06.712720 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:06.714052 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:06.718008 (+ 3956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:06.718015 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:06.718019 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:06.718446 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:06.718909 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:06.718917 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:06.718921 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:06.719101 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:06.719650 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:06.719654 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:06.719901 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 19:42:06.720095 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 19:42:06.720459 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":378,"threads_started":1}
I20260806 19:42:06.721311 24161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:42:06.723770 24299 heartbeater.cc:347] Connected to a master server at 127.21.202.126:38963
I20260806 19:42:06.724064 24299 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:06.724924 24299 heartbeater.cc:511] Master 127.21.202.126:38963 requested a full tablet report, sending...
I20260806 19:42:06.727365 24082 ts_manager.cc:194] Registered new tserver with Master: 8492820db869485582cc539ba164fac6 (127.21.202.65:37029)
I20260806 19:42:06.727440 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 24161
I20260806 19:42:06.727646 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:42:06.728475 22313 external_mini_cluster.cc:1428] Setting key 714d86dfafa30073459d81e13dbcb6f0
I20260806 19:42:06.729511 24082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:43353
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:42:06.777516 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:38963
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:43883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:07.128568 24303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:07.128928 24303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:07.129093 24303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:07.136628 24303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:07.136787 24303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:07.136958 24303 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 19:42:07.137046 24303 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:42:07.137176 24303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:42:07.147797 24303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:43883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.202.126:38963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:07.149315 24303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:07.150653 24303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:07.163677 24309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:07.163643 24308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:07.164206 24311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:07.164721 24303 server_base.cc:1034] running on GCE node
I20260806 19:42:07.165452 24303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:07.166410 24303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:07.167678 24303 hybrid_clock.cc:648] HybridClock initialized: now 1786045327167627 us; error 57 us; skew 500 ppm
I20260806 19:42:07.168066 24303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:07 dist-test-slave-77v9 krb5kdc[24037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045327, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:07.171669 24303 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:42:07.173200 24303 webserver.cc:460] Webserver started at http://127.21.202.66:45979/ using document root <none> and password file <none>
I20260806 19:42:07.173915 24303 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:07.174062 24303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:07.174392 24303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:07.177393 24303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "30ef7444fa3f4a869f9abeefca5b0a5c"
format_stamp: "Formatted at 2026-08-06 19:42:07 on dist-test-slave-77v9"
server_key: "5e05ac659a86dc6736caa16a1ac30485"
server_key_iv: "5cf37a2206c4d1193451978b416abaa9"
server_key_version: "encryptionkey@0"
I20260806 19:42:07.178164 24303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "30ef7444fa3f4a869f9abeefca5b0a5c"
format_stamp: "Formatted at 2026-08-06 19:42:07 on dist-test-slave-77v9"
server_key: "5e05ac659a86dc6736caa16a1ac30485"
server_key_iv: "5cf37a2206c4d1193451978b416abaa9"
server_key_version: "encryptionkey@0"
I20260806 19:42:07.183342 24303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:42:07.186935 24318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:07.188398 24303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:42:07.188598 24303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "30ef7444fa3f4a869f9abeefca5b0a5c"
format_stamp: "Formatted at 2026-08-06 19:42:07 on dist-test-slave-77v9"
server_key: "5e05ac659a86dc6736caa16a1ac30485"
server_key_iv: "5cf37a2206c4d1193451978b416abaa9"
server_key_version: "encryptionkey@0"
I20260806 19:42:07.188894 24303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:07.214533 24303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:07.219066 24303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:07.219395 24303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:07.222574 24303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:07.222702 24303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:07.222827 24303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:07.223093 24303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 06 19:42:07 dist-test-slave-77v9 krb5kdc[24037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045327, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:07.247463 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.223255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:53733 (local address 127.21.202.126:38963)
0806 19:42:07.224177 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:07.224181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:07.225192 (+ 1011us) server_negotiation.cc:407] Connection header received
0806 19:42:07.227400 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:07.227406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:07.227503 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:07.227622 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:07.229850 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.230983 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.233752 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.234264 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.234425 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.239976 (+ 5551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:07.239981 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:07.239984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:07.240018 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:07.243725 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:07.244767 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:07.244772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:07.244776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:07.244991 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:07.245610 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:07.245618 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:07.245623 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:07.245939 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:07.246111 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:07.246839 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 19:42:07.247069 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":694,"threads_started":1}
I20260806 19:42:07.248482 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.224050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:38963 (local address 127.21.202.66:53733)
0806 19:42:07.225016 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:07.225068 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:07.227113 (+ 2045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:07.227890 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:07.227898 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:07.228706 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:07.229662 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.229681 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.231133 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.231139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.233588 (+ 2449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.233609 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.234594 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.234605 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.235016 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.236008 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:07.236053 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:07.239733 (+ 3680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:07.243913 (+ 4180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:07.243926 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:07.243948 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:07.244632 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:07.245151 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:07.245157 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:07.245165 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:07.245419 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:07.246073 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:07.246081 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:07.246459 (+ 378us) client_negotiation.cc:769] Sending connection context
0806 19:42:07.246734 (+ 275us) client_negotiation.cc:240] Negotiation successful
0806 19:42:07.247267 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":387,"threads_started":1}
I20260806 19:42:07.273330 24303 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:34313
I20260806 19:42:07.273371 24440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:34313 every 8 connection(s)
I20260806 19:42:07.273460 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.256000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:38963 (local address 127.21.202.66:51815)
0806 19:42:07.256667 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:07.256690 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:07.256921 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:07.257528 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:07.257538 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:07.258345 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:07.259052 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.259073 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.260545 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.260551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.262045 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.262060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.263408 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.263413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.263635 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.264497 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:07.264536 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:07.266156 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:07.269753 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:07.269760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:07.269767 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:07.270499 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:07.271070 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:07.271075 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:07.271079 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:07.271281 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:07.272015 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:07.272020 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:07.272272 (+ 252us) client_negotiation.cc:769] Sending connection context
0806 19:42:07.272475 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 19:42:07.272898 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":382,"threads_started":1}
I20260806 19:42:07.273566 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.255943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:51815 (local address 127.21.202.126:38963)
0806 19:42:07.256116 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:07.256120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:07.256852 (+ 732us) server_negotiation.cc:407] Connection header received
0806 19:42:07.257081 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:07.257092 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:07.257170 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:07.257293 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:07.259252 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.260396 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.262236 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.262915 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.263093 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.266400 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:07.266412 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:07.266416 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:07.266465 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:07.269530 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:07.270705 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:07.270709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:07.270712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:07.270925 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:07.271440 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:07.271444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:07.271446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:07.271843 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:07.272055 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:07.273007 (+ 952us) server_negotiation.cc:299] Negotiation successful
0806 19:42:07.273254 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 19:42:07.274781 24303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:42:07.277982 24441 heartbeater.cc:347] Connected to a master server at 127.21.202.126:38963
I20260806 19:42:07.278255 24441 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:07.278913 24441 heartbeater.cc:511] Master 127.21.202.126:38963 requested a full tablet report, sending...
I20260806 19:42:07.280351 24081 ts_manager.cc:194] Registered new tserver with Master: 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313)
I20260806 19:42:07.281316 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 24303
I20260806 19:42:07.281503 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:42:07.281518 24081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:53733
I20260806 19:42:07.282145 22313 external_mini_cluster.cc:1428] Setting key 742f864fb0acf64d1ce08b4030e92eaf
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:42:07.333254 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:38963
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:43883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:07.673497 24445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:07.673785 24445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:07.673923 24445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:07.681275 24445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:07.681393 24445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:07.681538 24445 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 19:42:07.681613 24445 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:42:07.681740 24445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:42:07.692013 24445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:43883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.202.126:38963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:07.693667 24445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:07.695183 24445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:07.708629 24450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:07.708756 24453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:07.708655 24451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:07.709882 24445 server_base.cc:1034] running on GCE node
I20260806 19:42:07.710623 24445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:07.711650 24445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:07.712963 24445 hybrid_clock.cc:648] HybridClock initialized: now 1786045327712892 us; error 66 us; skew 500 ppm
I20260806 19:42:07.713316 24445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:07 dist-test-slave-77v9 krb5kdc[24037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045327, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:07.721637 24445 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:42:07.723469 24445 webserver.cc:460] Webserver started at http://127.21.202.67:35367/ using document root <none> and password file <none>
I20260806 19:42:07.724231 24445 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:07.724377 24445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:07.724700 24445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:07.727664 24445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946"
format_stamp: "Formatted at 2026-08-06 19:42:07 on dist-test-slave-77v9"
server_key: "ff0d596cbfae91646e92467e624baf20"
server_key_iv: "321e0539d422eac8b65cecfbf543eb95"
server_key_version: "encryptionkey@0"
I20260806 19:42:07.728622 24445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946"
format_stamp: "Formatted at 2026-08-06 19:42:07 on dist-test-slave-77v9"
server_key: "ff0d596cbfae91646e92467e624baf20"
server_key_iv: "321e0539d422eac8b65cecfbf543eb95"
server_key_version: "encryptionkey@0"
I20260806 19:42:07.732576 24299 heartbeater.cc:503] Master 127.21.202.126:38963 was elected leader, sending a full tablet report...
I20260806 19:42:07.735109 24445 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20260806 19:42:07.739017 24460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:07.740271 24445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:42:07.740456 24445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946"
format_stamp: "Formatted at 2026-08-06 19:42:07 on dist-test-slave-77v9"
server_key: "ff0d596cbfae91646e92467e624baf20"
server_key_iv: "321e0539d422eac8b65cecfbf543eb95"
server_key_version: "encryptionkey@0"
I20260806 19:42:07.740655 24445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:07.753933 24445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:07.758183 24445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:07.758489 24445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:07.761250 24445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:07.761384 24445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:07.761670 24445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:07.761878 24445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:42:07 dist-test-slave-77v9 krb5kdc[24037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045327, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:07.790064 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.762193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:34839 (local address 127.21.202.126:38963)
0806 19:42:07.762462 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:07.762469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:07.764269 (+ 1800us) server_negotiation.cc:407] Connection header received
0806 19:42:07.766809 (+ 2540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:07.766816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:07.766944 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:07.767072 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:07.770783 (+ 3711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.771761 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.774430 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.775100 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.775265 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.780188 (+ 4923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:07.780199 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:07.780208 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:07.780284 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:07.785050 (+ 4766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:07.786409 (+ 1359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:07.786414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:07.786417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:07.786661 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:07.787496 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:07.787505 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:07.787511 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:07.787991 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:07.788192 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:07.789179 (+ 987us) server_negotiation.cc:299] Negotiation successful
0806 19:42:07.789524 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 19:42:07.790889 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.763112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:38963 (local address 127.21.202.67:34839)
0806 19:42:07.764101 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:07.764141 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:07.766477 (+ 2336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:07.767564 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:07.767581 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:07.768777 (+ 1196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:07.770374 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.770411 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.772028 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.772035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.774276 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.774292 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.775536 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.775540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.775727 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.776579 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:07.776613 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:07.779853 (+ 3240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:07.785280 (+ 5427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:07.785296 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:07.785328 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:07.786205 (+ 877us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:07.786999 (+ 794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:07.787005 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:07.787009 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:07.787326 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:07.788148 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:07.788155 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:07.788608 (+ 453us) client_negotiation.cc:769] Sending connection context
0806 19:42:07.789035 (+ 427us) client_negotiation.cc:240] Negotiation successful
0806 19:42:07.789571 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":382,"threads_started":1}
I20260806 19:42:07.811347 24445 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:43657
I20260806 19:42:07.811446 24580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:43657 every 8 connection(s)
I20260806 19:42:07.812749 24445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:42:07.815222 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 24445
I20260806 19:42:07.815344 24581 heartbeater.cc:347] Connected to a master server at 127.21.202.126:38963
I20260806 19:42:07.815428 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:42:07.815613 24581 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:07.816038 22313 external_mini_cluster.cc:1428] Setting key d52773469584bb4e44b86c544861850a
I20260806 19:42:07.816572 24581 heartbeater.cc:511] Master 127.21.202.126:38963 requested a full tablet report, sending...
I20260806 19:42:07.816802 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.799884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:49903 (local address 127.21.202.126:38963)
0806 19:42:07.800121 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:07.800127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:07.800802 (+ 675us) server_negotiation.cc:407] Connection header received
0806 19:42:07.801110 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:07.801119 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:07.801243 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:07.801372 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:07.803280 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.804481 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.806485 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.807108 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.807260 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.809967 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:07.809976 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:07.809984 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:07.810039 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:07.813138 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:07.814144 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:07.814150 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:07.814153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:07.814373 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:07.814935 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:07.814940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:07.814944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:07.815369 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:07.815521 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:07.816091 (+ 570us) server_negotiation.cc:299] Negotiation successful
0806 19:42:07.816405 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 19:42:07.817268 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.799910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:38963 (local address 127.21.202.67:49903)
0806 19:42:07.800673 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:07.800701 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:07.800933 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:07.801569 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:07.801573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:07.802438 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:07.802996 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.803010 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.804722 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.804728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.806289 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.806304 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.807353 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.807361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.807690 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.808545 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:07.808566 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:07.809711 (+ 1145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:07.813301 (+ 3590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:07.813307 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:07.813311 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:07.813963 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:07.814615 (+ 652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:07.814621 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:07.814624 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:07.814789 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:07.815589 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:07.815593 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:07.815819 (+ 226us) client_negotiation.cc:769] Sending connection context
0806 19:42:07.815981 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 19:42:07.816382 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":368,"threads_started":1}
I20260806 19:42:07.818383 24082 ts_manager.cc:194] Registered new tserver with Master: d1c2aa31b4e747eb8e628d1c2b0e2946 (127.21.202.67:43657)
I20260806 19:42:07.819475 24082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:34839
I20260806 19:42:07.819849 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 19:42:07 dist-test-slave-77v9 krb5kdc[24037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 19:42:07 dist-test-slave-77v9 krb5kdc[24037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:07.877334 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.860744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56822 (local address 127.21.202.126:38963)
0806 19:42:07.861059 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:07.861064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:07.861284 (+ 220us) server_negotiation.cc:407] Connection header received
0806 19:42:07.861668 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:07.861690 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:07.861821 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:07.861963 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:07.863475 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.864268 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.865637 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.866123 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.866232 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.871262 (+ 5030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:07.871268 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:07.871271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:07.871314 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:07.874564 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:07.875261 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:07.875265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:07.875268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:07.875390 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:07.875818 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:07.875822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:07.875825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:07.876153 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:07.876391 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:07.876782 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 19:42:07.877056 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260806 19:42:07.882452 24082 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:42:07.889614 24082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:42:07.942929 24598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.905407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40934 (local address 127.21.202.66:34313)
0806 19:42:07.933321 (+ 27914us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:07.933330 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:07.933361 (+ 31us) server_negotiation.cc:407] Connection header received
0806 19:42:07.933458 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:07.933463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:07.933652 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:07.933876 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:07.935104 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.936501 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.939857 (+ 3356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.941409 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.941562 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.941792 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:07.942405 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 19:42:07.942601 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27730,"thread_start_us":27594,"threads_started":1}
I20260806 19:42:07.943199 24594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.904788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:34313 (local address 127.0.0.1:40934)
0806 19:42:07.932132 (+ 27344us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:07.932163 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:07.932314 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:07.934236 (+ 1922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:07.934241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:07.934277 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:07.934848 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.934864 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.936664 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.936670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.939657 (+ 2987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.939674 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.941698 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.941708 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.941986 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.942063 (+ 77us) client_negotiation.cc:769] Sending connection context
0806 19:42:07.942363 (+ 300us) client_negotiation.cc:240] Negotiation successful
0806 19:42:07.942655 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27152,"mutex_wait_us":117,"thread_start_us":29436,"threads_started":1}
I20260806 19:42:07.943523 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.905738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36622 (local address 127.21.202.65:37029)
0806 19:42:07.929396 (+ 23658us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:07.929408 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:07.934706 (+ 5298us) server_negotiation.cc:407] Connection header received
0806 19:42:07.934996 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:07.935004 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:07.935206 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:07.935414 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:07.936034 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.937454 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.940304 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.942118 (+ 1814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.942205 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.942403 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:07.943043 (+ 640us) server_negotiation.cc:299] Negotiation successful
0806 19:42:07.943208 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23427,"thread_start_us":23086,"threads_started":1}
I20260806 19:42:07.944309 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.904748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:43657 (local address 127.0.0.1:48420)
0806 19:42:07.930205 (+ 25457us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:07.930271 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:07.930541 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:07.933862 (+ 3321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:07.933870 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:07.933915 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:07.934501 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.934516 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.937457 (+ 2941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.937463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.940009 (+ 2546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.940050 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.943263 (+ 3213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.943359 (+ 96us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.943525 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.943551 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 19:42:07.943672 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 19:42:07.943868 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25060,"mutex_wait_us":73,"thread_start_us":27002,"threads_started":1}
I20260806 19:42:07.944489 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.905013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48420 (local address 127.21.202.67:43657)
0806 19:42:07.933044 (+ 28031us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:07.933055 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:07.933097 (+ 42us) server_negotiation.cc:407] Connection header received
0806 19:42:07.933208 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:07.933221 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:07.933538 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:07.933839 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:07.934752 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.937302 (+ 2550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.940416 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.942515 (+ 2099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.942670 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.942935 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:07.943798 (+ 863us) server_negotiation.cc:299] Negotiation successful
0806 19:42:07.944013 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27793,"thread_start_us":27569,"threads_started":1}
I20260806 19:42:07.946053 24595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:07.905139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37029 (local address 127.0.0.1:36622)
0806 19:42:07.934559 (+ 29420us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:07.934590 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:07.934815 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:07.935397 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:07.935401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:07.935426 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:07.935822 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.935837 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.937563 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.937568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.940103 (+ 2535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:07.940120 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:07.942626 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:07.942633 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:07.942813 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:07.942853 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 19:42:07.944132 (+ 1279us) client_negotiation.cc:240] Negotiation successful
0806 19:42:07.944274 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29279,"mutex_wait_us":115,"thread_start_us":25670,"threads_started":1}
I20260806 19:42:07.947731 24510 tablet_service.cc:1467] Processing CreateTablet for tablet 4466c6b9974a403a86869433ea549130 (DEFAULT_TABLE table=test-table [id=ffe715d1986448c2ab947029fafcdaed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:07.947739 24370 tablet_service.cc:1467] Processing CreateTablet for tablet 4466c6b9974a403a86869433ea549130 (DEFAULT_TABLE table=test-table [id=ffe715d1986448c2ab947029fafcdaed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:07.949136 24510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4466c6b9974a403a86869433ea549130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:07.949143 24370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4466c6b9974a403a86869433ea549130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:07.953167 24227 tablet_service.cc:1467] Processing CreateTablet for tablet 4466c6b9974a403a86869433ea549130 (DEFAULT_TABLE table=test-table [id=ffe715d1986448c2ab947029fafcdaed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:07.954787 24227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4466c6b9974a403a86869433ea549130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:07.961993 24600 tablet_bootstrap.cc:492] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946: Bootstrap starting.
I20260806 19:42:07.965544 24601 tablet_bootstrap.cc:492] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c: Bootstrap starting.
I20260806 19:42:07.966121 24600 tablet_bootstrap.cc:654] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:07.967572 24600 log.cc:824] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:07.969178 24601 tablet_bootstrap.cc:654] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:07.970466 24601 log.cc:824] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:07.971073 24600 tablet_bootstrap.cc:492] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946: No bootstrap required, opened a new log
I20260806 19:42:07.971405 24603 tablet_bootstrap.cc:492] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6: Bootstrap starting.
I20260806 19:42:07.971423 24600 ts_tablet_manager.cc:1397] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260806 19:42:07.975090 24603 tablet_bootstrap.cc:654] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:07.976122 24603 log.cc:824] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:07.979260 24601 tablet_bootstrap.cc:492] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c: No bootstrap required, opened a new log
I20260806 19:42:07.979460 24603 tablet_bootstrap.cc:492] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6: No bootstrap required, opened a new log
I20260806 19:42:07.979593 24601 ts_tablet_manager.cc:1397] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260806 19:42:07.979729 24603 ts_tablet_manager.cc:1397] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 19:42:07.983559 24600 raft_consensus.cc:348] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } }
I20260806 19:42:07.984123 24600 raft_consensus.cc:374] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:07.984226 24600 raft_consensus.cc:729] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c2aa31b4e747eb8e628d1c2b0e2946, State: Initialized, Role: FOLLOWER
I20260806 19:42:07.985018 24600 consensus_queue.cc:260] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } }
I20260806 19:42:07.986209 24581 heartbeater.cc:503] Master 127.21.202.126:38963 was elected leader, sending a full tablet report...
I20260806 19:42:07.986990 24600 ts_tablet_manager.cc:1428] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260806 19:42:07.990265 24601 raft_consensus.cc:348] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } }
I20260806 19:42:07.990783 24601 raft_consensus.cc:374] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:07.990907 24601 raft_consensus.cc:729] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ef7444fa3f4a869f9abeefca5b0a5c, State: Initialized, Role: FOLLOWER
I20260806 19:42:07.991614 24601 consensus_queue.cc:260] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } }
I20260806 19:42:07.992414 24603 raft_consensus.cc:348] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } }
I20260806 19:42:07.992921 24603 raft_consensus.cc:374] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:07.993036 24603 raft_consensus.cc:729] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8492820db869485582cc539ba164fac6, State: Initialized, Role: FOLLOWER
I20260806 19:42:07.993090 24441 heartbeater.cc:503] Master 127.21.202.126:38963 was elected leader, sending a full tablet report...
I20260806 19:42:07.993924 24601 ts_tablet_manager.cc:1428] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c: Time spent starting tablet: real 0.014s user 0.016s sys 0.001s
I20260806 19:42:07.993999 24603 consensus_queue.cc:260] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } }
I20260806 19:42:07.995796 24603 ts_tablet_manager.cc:1428] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6: Time spent starting tablet: real 0.016s user 0.012s sys 0.005s
W20260806 19:42:08.027190 24442 tablet.cc:2369] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:42:08.064675 24582 tablet.cc:2369] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:42:08.157718 24608 raft_consensus.cc:482] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:08.158139 24608 raft_consensus.cc:504] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } }
I20260806 19:42:08.159976 24608 leader_election.cc:288] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c2aa31b4e747eb8e628d1c2b0e2946 (127.21.202.67:43657), 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313)
I20260806 19:42:08.167595 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.160539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:48487 (local address 127.21.202.67:43657)
0806 19:42:08.160790 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:08.160800 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:08.161602 (+ 802us) server_negotiation.cc:407] Connection header received
0806 19:42:08.161823 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:08.161830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:08.161902 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:08.161972 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:08.162722 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.163438 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.165771 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.166553 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.166719 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.166815 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:08.167184 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 19:42:08.167310 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 19:42:08.167623 24598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.160571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:42453 (local address 127.21.202.66:34313)
0806 19:42:08.160847 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:08.160855 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:08.161689 (+ 834us) server_negotiation.cc:407] Connection header received
0806 19:42:08.161872 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:08.161876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:08.161941 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:08.162064 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:08.162757 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.163600 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.165683 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.166593 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.166806 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.166887 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:08.167145 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 19:42:08.167296 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 19:42:08.167832 24610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.160171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:43657 (local address 127.21.202.65:48487)
0806 19:42:08.161467 (+ 1296us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:08.161489 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:08.161656 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:08.162024 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:08.162030 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:08.162062 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:08.162568 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.162578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.163575 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.163580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.165607 (+ 2027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.165625 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.166801 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.166809 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.167003 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.167029 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 19:42:08.167142 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 19:42:08.167304 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1162,"mutex_wait_us":97,"thread_start_us":1045,"threads_started":1}
I20260806 19:42:08.168164 24609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.160551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:34313 (local address 127.21.202.65:42453)
0806 19:42:08.161535 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:08.161569 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:08.161709 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:08.162091 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:08.162095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:08.162123 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:08.162569 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.162578 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.163742 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.163746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.165506 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.165519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.166860 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.166865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.166988 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.167012 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 19:42:08.167095 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 19:42:08.167291 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":56,"thread_start_us":558,"threads_started":1}
I20260806 19:42:08.168995 24530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4466c6b9974a403a86869433ea549130" candidate_uuid: "8492820db869485582cc539ba164fac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" is_pre_election: true
I20260806 19:42:08.169415 24390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4466c6b9974a403a86869433ea549130" candidate_uuid: "8492820db869485582cc539ba164fac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" is_pre_election: true
I20260806 19:42:08.169656 24530 raft_consensus.cc:2454] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8492820db869485582cc539ba164fac6 in term 0.
I20260806 19:42:08.169953 24390 raft_consensus.cc:2454] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8492820db869485582cc539ba164fac6 in term 0.
I20260806 19:42:08.170468 24177 leader_election.cc:302] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8492820db869485582cc539ba164fac6, d1c2aa31b4e747eb8e628d1c2b0e2946; no voters:
I20260806 19:42:08.170976 24608 raft_consensus.cc:2781] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:42:08.171162 24608 raft_consensus.cc:482] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:08.171317 24608 raft_consensus.cc:3037] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:08.173985 24608 raft_consensus.cc:504] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } }
I20260806 19:42:08.174736 24608 leader_election.cc:288] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [CANDIDATE]: Term 1 election: Requested vote from peers d1c2aa31b4e747eb8e628d1c2b0e2946 (127.21.202.67:43657), 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313)
I20260806 19:42:08.175145 24530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4466c6b9974a403a86869433ea549130" candidate_uuid: "8492820db869485582cc539ba164fac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946"
I20260806 19:42:08.175374 24530 raft_consensus.cc:3037] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:08.175400 24390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4466c6b9974a403a86869433ea549130" candidate_uuid: "8492820db869485582cc539ba164fac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c"
I20260806 19:42:08.175681 24390 raft_consensus.cc:3037] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:08.177949 24530 raft_consensus.cc:2454] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8492820db869485582cc539ba164fac6 in term 1.
I20260806 19:42:08.178412 24390 raft_consensus.cc:2454] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8492820db869485582cc539ba164fac6 in term 1.
I20260806 19:42:08.178424 24177 leader_election.cc:302] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8492820db869485582cc539ba164fac6, d1c2aa31b4e747eb8e628d1c2b0e2946; no voters:
I20260806 19:42:08.178756 24608 raft_consensus.cc:2781] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:08.179275 24608 raft_consensus.cc:686] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [term 1 LEADER]: Becoming Leader. State: Replica: 8492820db869485582cc539ba164fac6, State: Running, Role: LEADER
I20260806 19:42:08.179785 24608 consensus_queue.cc:237] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } }
I20260806 19:42:08.185155 24081 catalog_manager.cc:5495] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8492820db869485582cc539ba164fac6 (127.21.202.65). New cstate: current_term: 1 leader_uuid: "8492820db869485582cc539ba164fac6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } health_report { overall_health: HEALTHY } } }
W20260806 19:42:08.224313 24300 tablet.cc:2369] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 19:42:08 dist-test-slave-77v9 krb5kdc[24037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045325, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:08.278950 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.263679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56824 (local address 127.21.202.126:38963)
0806 19:42:08.263959 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:08.263967 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:08.264655 (+ 688us) server_negotiation.cc:407] Connection header received
0806 19:42:08.264912 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:08.264916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:08.264995 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:08.265086 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:08.266421 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.267165 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.268520 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.268978 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.269082 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.272929 (+ 3847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:08.272935 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:08.272938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:08.272970 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:08.275816 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:08.276616 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:08.276624 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:08.276628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:08.276771 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:08.277229 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:08.277236 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:08.277239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:08.277549 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:08.277756 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:08.278211 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 19:42:08.278473 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":64}
I20260806 19:42:08.279110 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.263769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:38963 (local address 127.0.0.1:56824)
0806 19:42:08.264507 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:08.264526 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:08.264672 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:08.265164 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:08.265168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:08.265969 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:08.266259 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.266269 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.267300 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.267305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.268335 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.268347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.269154 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.269159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.269270 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.270039 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:08.270066 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:08.272682 (+ 2616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:08.275960 (+ 3278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:08.275977 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:08.275984 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:08.276489 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:08.276943 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:08.276949 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:08.276953 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:08.277110 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:08.277701 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:08.277712 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:08.277912 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 19:42:08.278095 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 19:42:08.278595 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":170,"thread_start_us":453,"threads_started":1}
I20260806 19:42:08.285014 24081 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:56824:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 19:42:08.286136 24081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:42:08.293998 24370 tablet_service.cc:1467] Processing CreateTablet for tablet bcdc8df7972e4326a45259833853defd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b55ffed9ef3548e3a31ffc31445604a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:42:08.293821 24227 tablet_service.cc:1467] Processing CreateTablet for tablet bcdc8df7972e4326a45259833853defd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b55ffed9ef3548e3a31ffc31445604a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:42:08.294080 24510 tablet_service.cc:1467] Processing CreateTablet for tablet bcdc8df7972e4326a45259833853defd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b55ffed9ef3548e3a31ffc31445604a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:42:08.294976 24370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcdc8df7972e4326a45259833853defd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:08.294963 24227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcdc8df7972e4326a45259833853defd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:08.295014 24510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcdc8df7972e4326a45259833853defd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:08.301422 24600 tablet_bootstrap.cc:492] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946: Bootstrap starting.
I20260806 19:42:08.302493 24603 tablet_bootstrap.cc:492] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6: Bootstrap starting.
I20260806 19:42:08.303131 24600 tablet_bootstrap.cc:654] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:08.304286 24603 tablet_bootstrap.cc:654] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:08.305508 24600 tablet_bootstrap.cc:492] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946: No bootstrap required, opened a new log
I20260806 19:42:08.305605 24601 tablet_bootstrap.cc:492] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c: Bootstrap starting.
I20260806 19:42:08.305670 24600 ts_tablet_manager.cc:1397] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260806 19:42:08.306301 24600 raft_consensus.cc:348] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.306542 24600 raft_consensus.cc:374] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:08.306646 24600 raft_consensus.cc:729] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c2aa31b4e747eb8e628d1c2b0e2946, State: Initialized, Role: FOLLOWER
I20260806 19:42:08.306901 24600 consensus_queue.cc:260] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.307566 24600 ts_tablet_manager.cc:1428] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 19:42:08.307425 24606 tablet_replica.cc:439] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } }
I20260806 19:42:08.307799 24601 tablet_bootstrap.cc:654] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:08.307798 24606 tablet_replica.cc:442] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:08.310351 24603 tablet_bootstrap.cc:492] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6: No bootstrap required, opened a new log
I20260806 19:42:08.310523 24603 ts_tablet_manager.cc:1397] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260806 19:42:08.311630 24603 raft_consensus.cc:348] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.311964 24603 raft_consensus.cc:374] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:08.312083 24603 raft_consensus.cc:729] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8492820db869485582cc539ba164fac6, State: Initialized, Role: FOLLOWER
I20260806 19:42:08.312367 24603 consensus_queue.cc:260] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.312923 24608 tablet_replica.cc:439] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } }
I20260806 19:42:08.313265 24608 tablet_replica.cc:442] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:08.313482 24603 ts_tablet_manager.cc:1428] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 19:42:08.315194 24601 tablet_bootstrap.cc:492] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c: No bootstrap required, opened a new log
I20260806 19:42:08.315366 24601 ts_tablet_manager.cc:1397] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260806 19:42:08.316030 24601 raft_consensus.cc:348] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.316260 24601 raft_consensus.cc:374] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:08.316359 24601 raft_consensus.cc:729] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ef7444fa3f4a869f9abeefca5b0a5c, State: Initialized, Role: FOLLOWER
I20260806 19:42:08.316614 24601 consensus_queue.cc:260] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.317250 24601 ts_tablet_manager.cc:1428] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 19:42:08.317117 24607 tablet_replica.cc:439] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } }
I20260806 19:42:08.317422 24607 tablet_replica.cc:442] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:08.446259 24608 raft_consensus.cc:482] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:08.446617 24608 raft_consensus.cc:504] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.447408 24608 leader_election.cc:288] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313), d1c2aa31b4e747eb8e628d1c2b0e2946 (127.21.202.67:43657)
I20260806 19:42:08.448030 24530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcdc8df7972e4326a45259833853defd" candidate_uuid: "8492820db869485582cc539ba164fac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" is_pre_election: true
I20260806 19:42:08.448318 24530 raft_consensus.cc:2454] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8492820db869485582cc539ba164fac6 in term 0.
I20260806 19:42:08.448279 24390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcdc8df7972e4326a45259833853defd" candidate_uuid: "8492820db869485582cc539ba164fac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" is_pre_election: true
I20260806 19:42:08.448554 24390 raft_consensus.cc:2454] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8492820db869485582cc539ba164fac6 in term 0.
I20260806 19:42:08.448863 24177 leader_election.cc:302] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8492820db869485582cc539ba164fac6, d1c2aa31b4e747eb8e628d1c2b0e2946; no voters:
I20260806 19:42:08.449200 24608 raft_consensus.cc:2781] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:42:08.449332 24608 raft_consensus.cc:482] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:08.449473 24608 raft_consensus.cc:3037] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:08.452529 24608 raft_consensus.cc:504] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.453521 24608 leader_election.cc:288] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [CANDIDATE]: Term 1 election: Requested vote from peers 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313), d1c2aa31b4e747eb8e628d1c2b0e2946 (127.21.202.67:43657)
I20260806 19:42:08.454012 24390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcdc8df7972e4326a45259833853defd" candidate_uuid: "8492820db869485582cc539ba164fac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c"
I20260806 19:42:08.454046 24530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcdc8df7972e4326a45259833853defd" candidate_uuid: "8492820db869485582cc539ba164fac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946"
I20260806 19:42:08.454257 24390 raft_consensus.cc:3037] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:08.454320 24530 raft_consensus.cc:3037] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:08.456609 24390 raft_consensus.cc:2454] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8492820db869485582cc539ba164fac6 in term 1.
I20260806 19:42:08.456817 24530 raft_consensus.cc:2454] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8492820db869485582cc539ba164fac6 in term 1.
I20260806 19:42:08.457258 24180 leader_election.cc:302] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ef7444fa3f4a869f9abeefca5b0a5c, 8492820db869485582cc539ba164fac6; no voters:
I20260806 19:42:08.457643 24608 raft_consensus.cc:2781] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:08.457852 24608 raft_consensus.cc:686] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [term 1 LEADER]: Becoming Leader. State: Replica: 8492820db869485582cc539ba164fac6, State: Running, Role: LEADER
I20260806 19:42:08.458184 24608 consensus_queue.cc:237] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.460065 24611 tablet_replica.cc:439] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6: TxnStatusTablet state changed. Reason: New leader 8492820db869485582cc539ba164fac6. Latest consensus state: current_term: 1 leader_uuid: "8492820db869485582cc539ba164fac6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } }
I20260806 19:42:08.460426 24611 tablet_replica.cc:442] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:42:08.461256 24624 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 19:42:08.461622 24624 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 19:42:08.461737 24081 catalog_manager.cc:5495] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8492820db869485582cc539ba164fac6 (127.21.202.65). New cstate: current_term: 1 leader_uuid: "8492820db869485582cc539ba164fac6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } health_report { overall_health: UNKNOWN } } }
I20260806 19:42:08.502734 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.496387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37029 (local address 127.0.0.1:36632)
0806 19:42:08.496600 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:08.496624 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:08.496781 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:08.497527 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:08.497532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:08.497546 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:42:08.497757 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.497785 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.499063 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.499070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.500178 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.500190 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.500951 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.500957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.501125 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.501182 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:42:08.502138 (+ 956us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:42:08.502162 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 19:42:08.502247 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 19:42:08.502412 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 19:42:08.502816 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.496463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36632 (local address 127.21.202.65:37029)
0806 19:42:08.497124 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:08.497130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:08.497145 (+ 15us) server_negotiation.cc:407] Connection header received
0806 19:42:08.497223 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:08.497227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:08.497324 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:08.497496 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:08.497969 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.498926 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.500320 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.500740 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.500926 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.501418 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:08.502027 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:08.502182 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:08.502286 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 19:42:08.502369 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":280,"threads_started":1}
I20260806 19:42:08.511873 24390 raft_consensus.cc:1261] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Refusing update from remote peer 8492820db869485582cc539ba164fac6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:08.511823 24530 raft_consensus.cc:1261] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Refusing update from remote peer 8492820db869485582cc539ba164fac6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:08.512957 24608 consensus_queue.cc:1035] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:08.513309 24611 consensus_queue.cc:1035] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:08.525820 24607 tablet_replica.cc:439] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c: TxnStatusTablet state changed. Reason: New leader 8492820db869485582cc539ba164fac6. Latest consensus state: current_term: 1 leader_uuid: "8492820db869485582cc539ba164fac6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } }
I20260806 19:42:08.526254 24607 tablet_replica.cc:442] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:08.526580 24606 tablet_replica.cc:439] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946: TxnStatusTablet state changed. Reason: New leader 8492820db869485582cc539ba164fac6. Latest consensus state: current_term: 1 leader_uuid: "8492820db869485582cc539ba164fac6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } }
I20260806 19:42:08.526925 24606 tablet_replica.cc:442] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:08.532559 24628 mvcc.cc:205] Tried to move back new op lower bound from 7315641665573097472 to 7315641665367248896. Current Snapshot: MvccSnapshot[applied={T|T < 7315641665573097472}]
I20260806 19:42:08.532858 24629 mvcc.cc:205] Tried to move back new op lower bound from 7315641665573097472 to 7315641665367248896. Current Snapshot: MvccSnapshot[applied={T|T < 7315641665573097472}]
I20260806 19:42:08.536049 24606 tablet_replica.cc:439] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8492820db869485582cc539ba164fac6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } }
I20260806 19:42:08.536435 24606 tablet_replica.cc:442] T bcdc8df7972e4326a45259833853defd P d1c2aa31b4e747eb8e628d1c2b0e2946: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:08.539949 24608 tablet_replica.cc:439] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8492820db869485582cc539ba164fac6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } }
I20260806 19:42:08.540370 24608 tablet_replica.cc:442] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:42:08.540907 24608 tablet_replica.cc:439] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8492820db869485582cc539ba164fac6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } }
I20260806 19:42:08.541213 24608 tablet_replica.cc:442] T bcdc8df7972e4326a45259833853defd P 8492820db869485582cc539ba164fac6: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:42:08.543000 24607 tablet_replica.cc:439] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8492820db869485582cc539ba164fac6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } } }
I20260806 19:42:08.543249 24607 tablet_replica.cc:442] T bcdc8df7972e4326a45259833853defd P 30ef7444fa3f4a869f9abeefca5b0a5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:42:08.557782 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.551531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36642 (local address 127.21.202.65:37029)
0806 19:42:08.551740 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:08.551748 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:08.552273 (+ 525us) server_negotiation.cc:407] Connection header received
0806 19:42:08.552512 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:08.552517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:08.552606 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:08.552724 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:08.553236 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.554126 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.555741 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.556259 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.556413 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.556964 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:08.557091 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:08.557196 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:08.557360 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 19:42:08.557446 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 19:42:08.578194 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.570646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:58965 (local address 127.21.202.65:37029)
0806 19:42:08.570880 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:08.570887 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:08.571589 (+ 702us) server_negotiation.cc:407] Connection header received
0806 19:42:08.571768 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:08.571773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:08.571856 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:08.571959 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:08.572632 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.573887 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.575775 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.576486 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.576611 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.577260 (+ 649us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:08.577398 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:08.577498 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:08.577706 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 19:42:08.577801 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 19:42:08.578287 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.570665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37029 (local address 127.21.202.65:58965)
0806 19:42:08.571484 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:08.571501 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:08.571638 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:08.572048 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:08.572052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:08.572062 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:42:08.572473 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.572486 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.574096 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.574101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.575610 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.575624 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.576703 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.576708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.576966 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.577031 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:42:08.577546 (+ 515us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:42:08.577559 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 19:42:08.577651 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 19:42:08.577783 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":464,"threads_started":1}
I20260806 19:42:08.588307 24390 raft_consensus.cc:1261] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Refusing update from remote peer 8492820db869485582cc539ba164fac6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:08.588390 24530 raft_consensus.cc:1261] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Refusing update from remote peer 8492820db869485582cc539ba164fac6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:08.588920 24608 consensus_queue.cc:1035] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:08.589283 24611 consensus_queue.cc:1035] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:08.662627 24081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:08.662987 24081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:08.667233 24081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e: Sending DeleteTablet for 3 replicas of tablet 4466c6b9974a403a86869433ea549130
I20260806 19:42:08.668673 24510 tablet_service.cc:1514] Processing DeleteTablet for tablet 4466c6b9974a403a86869433ea549130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:08 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:48420
I20260806 19:42:08.669624 24645 tablet_replica.cc:330] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946: stopping tablet replica
I20260806 19:42:08.669938 24370 tablet_service.cc:1514] Processing DeleteTablet for tablet 4466c6b9974a403a86869433ea549130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:08 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:40934
I20260806 19:42:08.670147 24645 raft_consensus.cc:2229] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:08.670502 24645 raft_consensus.cc:2258] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:08.670835 24646 tablet_replica.cc:330] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c: stopping tablet replica
I20260806 19:42:08.671020 24227 tablet_service.cc:1514] Processing DeleteTablet for tablet 4466c6b9974a403a86869433ea549130 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:08 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:36622
I20260806 19:42:08.671353 24646 raft_consensus.cc:2229] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:08.671842 24646 raft_consensus.cc:2258] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:08.672713 24645 ts_tablet_manager.cc:1905] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:08.674044 24646 ts_tablet_manager.cc:1905] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:08.674304 24081 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:42:08.675130 24081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:42:08.680861 24645 ts_tablet_manager.cc:1918] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:42:08.681058 24645 log.cc:1196] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/4466c6b9974a403a86869433ea549130
I20260806 19:42:08.681685 24645 ts_tablet_manager.cc:1939] T 4466c6b9974a403a86869433ea549130 P d1c2aa31b4e747eb8e628d1c2b0e2946: Deleting consensus metadata
I20260806 19:42:08.683070 24646 ts_tablet_manager.cc:1918] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:42:08.683365 24646 log.cc:1196] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/4466c6b9974a403a86869433ea549130
I20260806 19:42:08.683859 24066 catalog_manager.cc:4841] TS d1c2aa31b4e747eb8e628d1c2b0e2946 (127.21.202.67:43657): tablet 4466c6b9974a403a86869433ea549130 (table test-table [id=ffe715d1986448c2ab947029fafcdaed]) successfully deleted
I20260806 19:42:08.684026 24646 ts_tablet_manager.cc:1939] T 4466c6b9974a403a86869433ea549130 P 30ef7444fa3f4a869f9abeefca5b0a5c: Deleting consensus metadata
I20260806 19:42:08.685952 24226 tablet_service.cc:1467] Processing CreateTablet for tablet cdf617eb086146269f14f9b60ebab5e3 (DEFAULT_TABLE table=test-table [id=6515545f9b854bbf9d57cebefeeeac11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:08.686488 24226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdf617eb086146269f14f9b60ebab5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:08.689163 24069 catalog_manager.cc:4841] TS 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313): tablet 4466c6b9974a403a86869433ea549130 (table test-table [id=ffe715d1986448c2ab947029fafcdaed]) successfully deleted
I20260806 19:42:08.689661 24510 tablet_service.cc:1467] Processing CreateTablet for tablet cdf617eb086146269f14f9b60ebab5e3 (DEFAULT_TABLE table=test-table [id=6515545f9b854bbf9d57cebefeeeac11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:08.690110 24510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdf617eb086146269f14f9b60ebab5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:08.691569 24370 tablet_service.cc:1467] Processing CreateTablet for tablet cdf617eb086146269f14f9b60ebab5e3 (DEFAULT_TABLE table=test-table [id=6515545f9b854bbf9d57cebefeeeac11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:08.692054 24370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdf617eb086146269f14f9b60ebab5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:08.695781 24647 tablet_replica.cc:330] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6: stopping tablet replica
I20260806 19:42:08.696465 24647 raft_consensus.cc:2229] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:42:08.697876 24647 raft_consensus.cc:2258] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:08.698365 24600 tablet_bootstrap.cc:492] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946: Bootstrap starting.
I20260806 19:42:08.700191 24600 tablet_bootstrap.cc:654] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:08.700598 24647 ts_tablet_manager.cc:1905] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:08.708249 24603 tablet_bootstrap.cc:492] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6: Bootstrap starting.
I20260806 19:42:08.709978 24647 ts_tablet_manager.cc:1918] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:42:08.710283 24647 log.cc:1196] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/4466c6b9974a403a86869433ea549130
I20260806 19:42:08.710623 24603 tablet_bootstrap.cc:654] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:08.710673 24600 tablet_bootstrap.cc:492] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946: No bootstrap required, opened a new log
I20260806 19:42:08.710857 24600 ts_tablet_manager.cc:1397] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.009s
I20260806 19:42:08.711796 24600 raft_consensus.cc:348] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.712087 24600 raft_consensus.cc:374] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:08.711983 24647 ts_tablet_manager.cc:1939] T 4466c6b9974a403a86869433ea549130 P 8492820db869485582cc539ba164fac6: Deleting consensus metadata
I20260806 19:42:08.712190 24600 raft_consensus.cc:729] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c2aa31b4e747eb8e628d1c2b0e2946, State: Initialized, Role: FOLLOWER
I20260806 19:42:08.712495 24600 consensus_queue.cc:260] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.713721 24600 ts_tablet_manager.cc:1428] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 19:42:08.714744 24601 tablet_bootstrap.cc:492] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c: Bootstrap starting.
I20260806 19:42:08.715906 24603 tablet_bootstrap.cc:492] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6: No bootstrap required, opened a new log
I20260806 19:42:08.716184 24603 ts_tablet_manager.cc:1397] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260806 19:42:08.716925 24601 tablet_bootstrap.cc:654] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:08.716889 24068 catalog_manager.cc:4841] TS 8492820db869485582cc539ba164fac6 (127.21.202.65:37029): tablet 4466c6b9974a403a86869433ea549130 (table test-table [id=ffe715d1986448c2ab947029fafcdaed]) successfully deleted
I20260806 19:42:08.717329 24603 raft_consensus.cc:348] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.718080 24603 raft_consensus.cc:374] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:08.718271 24603 raft_consensus.cc:729] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8492820db869485582cc539ba164fac6, State: Initialized, Role: FOLLOWER
I20260806 19:42:08.718768 24603 consensus_queue.cc:260] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.723367 24603 ts_tablet_manager.cc:1428] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260806 19:42:08.729564 24601 tablet_bootstrap.cc:492] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c: No bootstrap required, opened a new log
I20260806 19:42:08.729799 24601 ts_tablet_manager.cc:1397] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.001s
I20260806 19:42:08.730551 24601 raft_consensus.cc:348] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.730804 24601 raft_consensus.cc:374] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:08.730912 24601 raft_consensus.cc:729] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ef7444fa3f4a869f9abeefca5b0a5c, State: Initialized, Role: FOLLOWER
I20260806 19:42:08.731225 24601 consensus_queue.cc:260] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.731801 24601 ts_tablet_manager.cc:1428] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 19:42:08.764143 24606 raft_consensus.cc:482] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:08.764429 24606 raft_consensus.cc:504] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.766242 24606 leader_election.cc:288] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8492820db869485582cc539ba164fac6 (127.21.202.65:37029), 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313)
I20260806 19:42:08.778172 24658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.767915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:54337 (local address 127.21.202.66:34313)
0806 19:42:08.769556 (+ 1641us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:08.769564 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:08.770981 (+ 1417us) server_negotiation.cc:407] Connection header received
0806 19:42:08.771072 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:08.771076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:08.771139 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:08.771263 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:08.772030 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.773412 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.775874 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.776813 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.777037 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.777204 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:08.777578 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 19:42:08.777738 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1472,"thread_start_us":1260,"threads_started":1}
I20260806 19:42:08.778211 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.766884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:34313 (local address 127.21.202.67:54337)
0806 19:42:08.770680 (+ 3796us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:08.770709 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:08.770848 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:08.771287 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:08.771291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:08.771322 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:08.771764 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.771777 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.773586 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.773593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.775709 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.775721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.777028 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.777034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.777328 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.777359 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 19:42:08.777528 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 19:42:08.777684 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3657,"mutex_wait_us":4,"thread_start_us":3943,"threads_started":1}
I20260806 19:42:08.778244 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.767232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:50845 (local address 127.21.202.65:37029)
0806 19:42:08.767588 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:08.767604 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:08.770234 (+ 2630us) server_negotiation.cc:407] Connection header received
0806 19:42:08.770581 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:08.770587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:08.770676 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:08.770742 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:08.771550 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.772533 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.775255 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.776723 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.777067 (+ 344us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.777152 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:08.777730 (+ 578us) server_negotiation.cc:299] Negotiation successful
0806 19:42:08.777921 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260806 19:42:08.779119 24656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.766596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37029 (local address 127.21.202.67:50845)
0806 19:42:08.770110 (+ 3514us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:08.770133 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:08.770382 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:08.771008 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:08.771013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:08.771042 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:08.771368 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.771382 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.772781 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.772786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.775068 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.775086 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.777310 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.777317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.777529 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.777564 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 19:42:08.778517 (+ 953us) client_negotiation.cc:240] Negotiation successful
0806 19:42:08.778667 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3342,"mutex_wait_us":1,"thread_start_us":3345,"threads_started":1}
I20260806 19:42:08.779762 24389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdf617eb086146269f14f9b60ebab5e3" candidate_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" is_pre_election: true
I20260806 19:42:08.780190 24389 raft_consensus.cc:2454] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1c2aa31b4e747eb8e628d1c2b0e2946 in term 0.
I20260806 19:42:08.781023 24464 leader_election.cc:302] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ef7444fa3f4a869f9abeefca5b0a5c, d1c2aa31b4e747eb8e628d1c2b0e2946; no voters:
I20260806 19:42:08.781718 24606 raft_consensus.cc:2781] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:42:08.781879 24606 raft_consensus.cc:482] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:08.782006 24606 raft_consensus.cc:3037] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:08.781951 24247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdf617eb086146269f14f9b60ebab5e3" candidate_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8492820db869485582cc539ba164fac6" is_pre_election: true
I20260806 19:42:08.782728 24247 raft_consensus.cc:2454] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1c2aa31b4e747eb8e628d1c2b0e2946 in term 0.
I20260806 19:42:08.784957 24606 raft_consensus.cc:504] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.785661 24606 leader_election.cc:288] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [CANDIDATE]: Term 1 election: Requested vote from peers 8492820db869485582cc539ba164fac6 (127.21.202.65:37029), 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313)
I20260806 19:42:08.786460 24247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdf617eb086146269f14f9b60ebab5e3" candidate_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8492820db869485582cc539ba164fac6"
I20260806 19:42:08.786620 24389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdf617eb086146269f14f9b60ebab5e3" candidate_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c"
I20260806 19:42:08.786972 24247 raft_consensus.cc:3037] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:08.787283 24389 raft_consensus.cc:3037] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:08.790901 24389 raft_consensus.cc:2454] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c2aa31b4e747eb8e628d1c2b0e2946 in term 1.
I20260806 19:42:08.791208 24247 raft_consensus.cc:2454] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c2aa31b4e747eb8e628d1c2b0e2946 in term 1.
I20260806 19:42:08.791499 24464 leader_election.cc:302] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ef7444fa3f4a869f9abeefca5b0a5c, d1c2aa31b4e747eb8e628d1c2b0e2946; no voters:
I20260806 19:42:08.791915 24606 raft_consensus.cc:2781] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:08.792483 24606 raft_consensus.cc:686] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 LEADER]: Becoming Leader. State: Replica: d1c2aa31b4e747eb8e628d1c2b0e2946, State: Running, Role: LEADER
I20260806 19:42:08.792999 24606 consensus_queue.cc:237] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:08.797646 24081 catalog_manager.cc:5495] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1c2aa31b4e747eb8e628d1c2b0e2946 (127.21.202.67). New cstate: current_term: 1 leader_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } health_report { overall_health: HEALTHY } } }
I20260806 19:42:08.832996 24661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.824964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60432 (local address 127.21.202.67:43657)
0806 19:42:08.825609 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:08.825616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:08.825774 (+ 158us) server_negotiation.cc:407] Connection header received
0806 19:42:08.825992 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:08.825998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:08.826106 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:08.826304 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:08.826724 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.828003 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.829940 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.830693 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.830834 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.831362 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:08.832185 (+ 823us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:08.832350 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:08.832473 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 19:42:08.832563 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":463,"threads_started":1}
I20260806 19:42:08.854040 24664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.846780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37029 (local address 127.21.202.67:33947)
0806 19:42:08.847559 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:08.847579 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:08.847741 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:08.848167 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:08.848172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:08.848187 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:42:08.848515 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.848528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.849817 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.849823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.851478 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.851495 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.852546 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.852553 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.852760 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.852887 (+ 127us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:42:08.853350 (+ 463us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:42:08.853363 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 19:42:08.853461 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 19:42:08.853619 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":366,"threads_started":1}
I20260806 19:42:08.854023 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.846744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:33947 (local address 127.21.202.65:37029)
0806 19:42:08.846942 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:08.846949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:08.847687 (+ 738us) server_negotiation.cc:407] Connection header received
0806 19:42:08.847905 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:08.847912 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:08.848020 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:08.848127 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:08.848713 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.849657 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.851714 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.852336 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.852503 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.853074 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:08.853244 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:08.853324 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:08.853526 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 19:42:08.853618 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 19:42:08.865466 24389 raft_consensus.cc:1261] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Refusing update from remote peer d1c2aa31b4e747eb8e628d1c2b0e2946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:08.865834 24247 raft_consensus.cc:1261] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Refusing update from remote peer d1c2aa31b4e747eb8e628d1c2b0e2946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:08.866968 24659 consensus_queue.cc:1035] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 19:42:08.867702 24606 consensus_queue.cc:1035] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:08.910828 24661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.905052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:44851 (local address 127.21.202.67:43657)
0806 19:42:08.905336 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:08.905345 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:08.905367 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:42:08.905500 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:08.905506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:08.905660 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:08.905802 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:08.906175 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.907115 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.909108 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.909535 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.909676 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.910110 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:08.910236 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:08.910324 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:08.910447 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 19:42:08.910527 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 19:42:08.910877 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:08.904750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:43657 (local address 127.21.202.65:44851)
0806 19:42:08.905019 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:08.905043 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:08.905184 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:08.905819 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:08.905824 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:08.905837 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:42:08.906042 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.906052 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.907272 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.907277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.908955 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:08.908969 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:08.909701 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:08.909707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:08.909905 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:08.909948 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:42:08.910330 (+ 382us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:42:08.910345 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 19:42:08.910426 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 19:42:08.910578 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
W20260806 19:42:08.912886 24629 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 19:42:10.518141 24082 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:10.518653 24082 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:10.523437 24082 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e: Sending DeleteTablet for 3 replicas of tablet cdf617eb086146269f14f9b60ebab5e3
I20260806 19:42:10.524426 24226 tablet_service.cc:1514] Processing DeleteTablet for tablet cdf617eb086146269f14f9b60ebab5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:10 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:36622
I20260806 19:42:10.524538 24510 tablet_service.cc:1514] Processing DeleteTablet for tablet cdf617eb086146269f14f9b60ebab5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:10 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:48420
I20260806 19:42:10.524554 24370 tablet_service.cc:1514] Processing DeleteTablet for tablet cdf617eb086146269f14f9b60ebab5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:10 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:40934
I20260806 19:42:10.537782 24692 tablet_replica.cc:330] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c: stopping tablet replica
I20260806 19:42:10.537792 24691 tablet_replica.cc:330] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6: stopping tablet replica
I20260806 19:42:10.538053 24691 raft_consensus.cc:2229] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:10.538225 24692 raft_consensus.cc:2229] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:10.538467 24691 raft_consensus.cc:2258] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:10.538661 24692 raft_consensus.cc:2258] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:10.539712 24691 ts_tablet_manager.cc:1905] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:10.539762 24693 tablet_replica.cc:330] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946: stopping tablet replica
I20260806 19:42:10.539989 24693 raft_consensus.cc:2229] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:42:10.540544 24692 ts_tablet_manager.cc:1905] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:10.540661 24693 raft_consensus.cc:2258] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:10.542560 24693 ts_tablet_manager.cc:1905] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:10.546059 24692 ts_tablet_manager.cc:1918] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 19:42:10.546221 24692 log.cc:1196] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/cdf617eb086146269f14f9b60ebab5e3
I20260806 19:42:10.546710 24692 ts_tablet_manager.cc:1939] T cdf617eb086146269f14f9b60ebab5e3 P 30ef7444fa3f4a869f9abeefca5b0a5c: Deleting consensus metadata
I20260806 19:42:10.547931 24069 catalog_manager.cc:4841] TS 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313): tablet cdf617eb086146269f14f9b60ebab5e3 (table test-table [id=6515545f9b854bbf9d57cebefeeeac11]) successfully deleted
I20260806 19:42:10.548064 24691 ts_tablet_manager.cc:1918] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 19:42:10.548211 24691 log.cc:1196] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/cdf617eb086146269f14f9b60ebab5e3
I20260806 19:42:10.548792 24691 ts_tablet_manager.cc:1939] T cdf617eb086146269f14f9b60ebab5e3 P 8492820db869485582cc539ba164fac6: Deleting consensus metadata
I20260806 19:42:10.550107 24068 catalog_manager.cc:4841] TS 8492820db869485582cc539ba164fac6 (127.21.202.65:37029): tablet cdf617eb086146269f14f9b60ebab5e3 (table test-table [id=6515545f9b854bbf9d57cebefeeeac11]) successfully deleted
I20260806 19:42:10.550631 24693 ts_tablet_manager.cc:1918] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 19:42:10.550770 24693 log.cc:1196] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/cdf617eb086146269f14f9b60ebab5e3
I20260806 19:42:10.551199 24693 ts_tablet_manager.cc:1939] T cdf617eb086146269f14f9b60ebab5e3 P d1c2aa31b4e747eb8e628d1c2b0e2946: Deleting consensus metadata
I20260806 19:42:10.552364 24066 catalog_manager.cc:4841] TS d1c2aa31b4e747eb8e628d1c2b0e2946 (127.21.202.67:43657): tablet cdf617eb086146269f14f9b60ebab5e3 (table test-table [id=6515545f9b854bbf9d57cebefeeeac11]) successfully deleted
Aug 06 19:42:10 dist-test-slave-77v9 krb5kdc[24037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045330, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 19:42:10 dist-test-slave-77v9 krb5kdc[24037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045330, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:10.601572 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:10.583440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35654 (local address 127.21.202.126:38963)
0806 19:42:10.584233 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:10.584244 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:10.584280 (+ 36us) server_negotiation.cc:407] Connection header received
0806 19:42:10.584569 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:10.584575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:10.584655 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:10.584808 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:10.586483 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:10.587374 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:10.588967 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:10.589477 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:10.589648 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:10.594945 (+ 5297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:10.594952 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:10.594955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:10.594999 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:10.598332 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:10.599187 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:10.599192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:10.599195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:10.599365 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:10.599933 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:10.599937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:10.599939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:10.600335 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:10.600597 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:10.601013 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 19:42:10.601227 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":652,"threads_started":1}
I20260806 19:42:10.605664 24082 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:42:10.606382 24082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:42:10.613703 24226 tablet_service.cc:1467] Processing CreateTablet for tablet 267eaf15f8b04e298d28dd36a160fa98 (DEFAULT_TABLE table=test-table [id=fd86ad9039d049679f427c21a6007df2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:10.613869 24370 tablet_service.cc:1467] Processing CreateTablet for tablet 267eaf15f8b04e298d28dd36a160fa98 (DEFAULT_TABLE table=test-table [id=fd86ad9039d049679f427c21a6007df2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:10.614236 24226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267eaf15f8b04e298d28dd36a160fa98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:10.614317 24370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267eaf15f8b04e298d28dd36a160fa98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:10.614398 24510 tablet_service.cc:1467] Processing CreateTablet for tablet 267eaf15f8b04e298d28dd36a160fa98 (DEFAULT_TABLE table=test-table [id=fd86ad9039d049679f427c21a6007df2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:10.614791 24510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267eaf15f8b04e298d28dd36a160fa98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:10.620738 24702 tablet_bootstrap.cc:492] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6: Bootstrap starting.
I20260806 19:42:10.621474 24703 tablet_bootstrap.cc:492] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c: Bootstrap starting.
I20260806 19:42:10.622283 24704 tablet_bootstrap.cc:492] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946: Bootstrap starting.
I20260806 19:42:10.622977 24702 tablet_bootstrap.cc:654] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:10.623318 24703 tablet_bootstrap.cc:654] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:10.624486 24704 tablet_bootstrap.cc:654] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:10.630918 24703 tablet_bootstrap.cc:492] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c: No bootstrap required, opened a new log
I20260806 19:42:10.631140 24703 ts_tablet_manager.cc:1397] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260806 19:42:10.631592 24704 tablet_bootstrap.cc:492] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946: No bootstrap required, opened a new log
I20260806 19:42:10.631773 24704 ts_tablet_manager.cc:1397] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260806 19:42:10.632042 24702 tablet_bootstrap.cc:492] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6: No bootstrap required, opened a new log
I20260806 19:42:10.632206 24702 ts_tablet_manager.cc:1397] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260806 19:42:10.632081 24703 raft_consensus.cc:348] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.632340 24703 raft_consensus.cc:374] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:10.632452 24703 raft_consensus.cc:729] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ef7444fa3f4a869f9abeefca5b0a5c, State: Initialized, Role: FOLLOWER
I20260806 19:42:10.632715 24703 consensus_queue.cc:260] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.632797 24704 raft_consensus.cc:348] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.633111 24704 raft_consensus.cc:374] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:10.633224 24704 raft_consensus.cc:729] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c2aa31b4e747eb8e628d1c2b0e2946, State: Initialized, Role: FOLLOWER
I20260806 19:42:10.633239 24702 raft_consensus.cc:348] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.633601 24702 raft_consensus.cc:374] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:10.633524 24704 consensus_queue.cc:260] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.633790 24702 raft_consensus.cc:729] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8492820db869485582cc539ba164fac6, State: Initialized, Role: FOLLOWER
I20260806 19:42:10.634135 24702 consensus_queue.cc:260] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.635035 24702 ts_tablet_manager.cc:1428] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:10.635902 24703 ts_tablet_manager.cc:1428] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 19:42:10.636488 24704 ts_tablet_manager.cc:1428] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260806 19:42:10.664124 24669 raft_consensus.cc:482] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:10.664355 24669 raft_consensus.cc:504] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.665244 24669 leader_election.cc:288] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8492820db869485582cc539ba164fac6 (127.21.202.65:37029), 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313)
I20260806 19:42:10.666074 24389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267eaf15f8b04e298d28dd36a160fa98" candidate_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" is_pre_election: true
I20260806 19:42:10.666069 24247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267eaf15f8b04e298d28dd36a160fa98" candidate_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8492820db869485582cc539ba164fac6" is_pre_election: true
I20260806 19:42:10.666363 24389 raft_consensus.cc:2454] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1c2aa31b4e747eb8e628d1c2b0e2946 in term 0.
I20260806 19:42:10.666385 24247 raft_consensus.cc:2454] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1c2aa31b4e747eb8e628d1c2b0e2946 in term 0.
I20260806 19:42:10.666886 24463 leader_election.cc:302] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8492820db869485582cc539ba164fac6, d1c2aa31b4e747eb8e628d1c2b0e2946; no voters:
I20260806 19:42:10.667255 24669 raft_consensus.cc:2781] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:42:10.667407 24669 raft_consensus.cc:482] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:10.667531 24669 raft_consensus.cc:3037] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:10.669632 24669 raft_consensus.cc:504] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.670241 24669 leader_election.cc:288] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [CANDIDATE]: Term 1 election: Requested vote from peers 8492820db869485582cc539ba164fac6 (127.21.202.65:37029), 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313)
I20260806 19:42:10.670750 24389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267eaf15f8b04e298d28dd36a160fa98" candidate_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c"
I20260806 19:42:10.670773 24247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267eaf15f8b04e298d28dd36a160fa98" candidate_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8492820db869485582cc539ba164fac6"
I20260806 19:42:10.670969 24389 raft_consensus.cc:3037] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:10.671003 24247 raft_consensus.cc:3037] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:10.673084 24389 raft_consensus.cc:2454] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c2aa31b4e747eb8e628d1c2b0e2946 in term 1.
I20260806 19:42:10.673105 24247 raft_consensus.cc:2454] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c2aa31b4e747eb8e628d1c2b0e2946 in term 1.
I20260806 19:42:10.673552 24464 leader_election.cc:302] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ef7444fa3f4a869f9abeefca5b0a5c, d1c2aa31b4e747eb8e628d1c2b0e2946; no voters:
I20260806 19:42:10.673888 24669 raft_consensus.cc:2781] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:10.674055 24669 raft_consensus.cc:686] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 LEADER]: Becoming Leader. State: Replica: d1c2aa31b4e747eb8e628d1c2b0e2946, State: Running, Role: LEADER
I20260806 19:42:10.674350 24669 consensus_queue.cc:237] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.677672 24080 catalog_manager.cc:5495] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1c2aa31b4e747eb8e628d1c2b0e2946 (127.21.202.67). New cstate: current_term: 1 leader_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } health_report { overall_health: HEALTHY } } }
I20260806 19:42:10.719579 24711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:10.712105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60440 (local address 127.21.202.67:43657)
0806 19:42:10.712725 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:10.712729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:10.712747 (+ 18us) server_negotiation.cc:407] Connection header received
0806 19:42:10.712852 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:10.712856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:10.712946 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:10.713121 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:10.713848 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:10.715231 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:10.717637 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:10.718117 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:10.718262 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:10.718713 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:10.718821 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:10.718922 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:10.719103 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 19:42:10.719188 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":353,"threads_started":1}
I20260806 19:42:10.730383 24247 raft_consensus.cc:1261] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Refusing update from remote peer d1c2aa31b4e747eb8e628d1c2b0e2946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:10.730793 24389 raft_consensus.cc:1261] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Refusing update from remote peer d1c2aa31b4e747eb8e628d1c2b0e2946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:10.730983 24669 consensus_queue.cc:1035] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:10.731364 24683 consensus_queue.cc:1035] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:10.781066 24080 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:10.781337 24080 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:10.785065 24080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e: Sending DeleteTablet for 3 replicas of tablet 267eaf15f8b04e298d28dd36a160fa98
I20260806 19:42:10.785993 24226 tablet_service.cc:1514] Processing DeleteTablet for tablet 267eaf15f8b04e298d28dd36a160fa98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:10 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:36622
I20260806 19:42:10.786012 24370 tablet_service.cc:1514] Processing DeleteTablet for tablet 267eaf15f8b04e298d28dd36a160fa98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:10 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:40934
I20260806 19:42:10.786175 24510 tablet_service.cc:1514] Processing DeleteTablet for tablet 267eaf15f8b04e298d28dd36a160fa98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:10 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:48420
I20260806 19:42:10.786358 24692 tablet_replica.cc:330] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c: stopping tablet replica
I20260806 19:42:10.786568 24692 raft_consensus.cc:2229] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:10.786796 24692 raft_consensus.cc:2258] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:10.787114 24693 tablet_replica.cc:330] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946: stopping tablet replica
I20260806 19:42:10.787377 24693 raft_consensus.cc:2229] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:42:10.787678 24693 raft_consensus.cc:2258] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:10.787667 24080 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 19:42:10.788118 24692 ts_tablet_manager.cc:1905] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 19:42:10.788525 24080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:42:10.788798 24693 ts_tablet_manager.cc:1905] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:10.793619 24691 tablet_replica.cc:330] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6: stopping tablet replica
I20260806 19:42:10.793893 24691 raft_consensus.cc:2229] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:10.794126 24691 raft_consensus.cc:2258] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:10.795240 24692 ts_tablet_manager.cc:1918] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:42:10.795433 24692 log.cc:1196] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/267eaf15f8b04e298d28dd36a160fa98
I20260806 19:42:10.795897 24692 ts_tablet_manager.cc:1939] T 267eaf15f8b04e298d28dd36a160fa98 P 30ef7444fa3f4a869f9abeefca5b0a5c: Deleting consensus metadata
I20260806 19:42:10.796680 24693 ts_tablet_manager.cc:1918] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:42:10.796892 24693 log.cc:1196] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/267eaf15f8b04e298d28dd36a160fa98
I20260806 19:42:10.797103 24370 tablet_service.cc:1467] Processing CreateTablet for tablet dd45e4cabb8d4496a1abc743e1320573 (DEFAULT_TABLE table=test-table [id=b9306e63fae446839f793272043344a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:10.797289 24226 tablet_service.cc:1467] Processing CreateTablet for tablet dd45e4cabb8d4496a1abc743e1320573 (DEFAULT_TABLE table=test-table [id=b9306e63fae446839f793272043344a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:10.797380 24510 tablet_service.cc:1467] Processing CreateTablet for tablet dd45e4cabb8d4496a1abc743e1320573 (DEFAULT_TABLE table=test-table [id=b9306e63fae446839f793272043344a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:10.797499 24693 ts_tablet_manager.cc:1939] T 267eaf15f8b04e298d28dd36a160fa98 P d1c2aa31b4e747eb8e628d1c2b0e2946: Deleting consensus metadata
I20260806 19:42:10.797539 24370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd45e4cabb8d4496a1abc743e1320573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:10.797837 24510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd45e4cabb8d4496a1abc743e1320573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:10.797860 24226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd45e4cabb8d4496a1abc743e1320573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:10.799479 24691 ts_tablet_manager.cc:1905] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:10.801173 24066 catalog_manager.cc:4841] TS d1c2aa31b4e747eb8e628d1c2b0e2946 (127.21.202.67:43657): tablet 267eaf15f8b04e298d28dd36a160fa98 (table test-table [id=fd86ad9039d049679f427c21a6007df2]) successfully deleted
I20260806 19:42:10.804508 24703 tablet_bootstrap.cc:492] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c: Bootstrap starting.
I20260806 19:42:10.804642 24704 tablet_bootstrap.cc:492] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946: Bootstrap starting.
I20260806 19:42:10.806288 24703 tablet_bootstrap.cc:654] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:10.806288 24704 tablet_bootstrap.cc:654] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:10.806479 24691 ts_tablet_manager.cc:1918] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:42:10.806675 24691 log.cc:1196] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/267eaf15f8b04e298d28dd36a160fa98
I20260806 19:42:10.807152 24691 ts_tablet_manager.cc:1939] T 267eaf15f8b04e298d28dd36a160fa98 P 8492820db869485582cc539ba164fac6: Deleting consensus metadata
I20260806 19:42:10.808343 24068 catalog_manager.cc:4841] TS 8492820db869485582cc539ba164fac6 (127.21.202.65:37029): tablet 267eaf15f8b04e298d28dd36a160fa98 (table test-table [id=fd86ad9039d049679f427c21a6007df2]) successfully deleted
I20260806 19:42:10.810086 24702 tablet_bootstrap.cc:492] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6: Bootstrap starting.
I20260806 19:42:10.811882 24702 tablet_bootstrap.cc:654] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:10.812242 24069 catalog_manager.cc:4841] TS 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313): tablet 267eaf15f8b04e298d28dd36a160fa98 (table test-table [id=fd86ad9039d049679f427c21a6007df2]) successfully deleted
I20260806 19:42:10.818123 24703 tablet_bootstrap.cc:492] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c: No bootstrap required, opened a new log
I20260806 19:42:10.818501 24703 ts_tablet_manager.cc:1397] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260806 19:42:10.819671 24703 raft_consensus.cc:348] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.820154 24703 raft_consensus.cc:374] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:10.820309 24703 raft_consensus.cc:729] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ef7444fa3f4a869f9abeefca5b0a5c, State: Initialized, Role: FOLLOWER
I20260806 19:42:10.820765 24703 consensus_queue.cc:260] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.821702 24703 ts_tablet_manager.cc:1428] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 19:42:10.823218 24702 tablet_bootstrap.cc:492] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6: No bootstrap required, opened a new log
I20260806 19:42:10.823415 24702 ts_tablet_manager.cc:1397] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.002s
I20260806 19:42:10.824239 24702 raft_consensus.cc:348] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.824542 24702 raft_consensus.cc:374] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:10.824621 24702 raft_consensus.cc:729] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8492820db869485582cc539ba164fac6, State: Initialized, Role: FOLLOWER
I20260806 19:42:10.824657 24704 tablet_bootstrap.cc:492] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946: No bootstrap required, opened a new log
I20260806 19:42:10.824867 24704 ts_tablet_manager.cc:1397] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.011s
I20260806 19:42:10.824945 24702 consensus_queue.cc:260] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.825560 24702 ts_tablet_manager.cc:1428] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 19:42:10.825870 24704 raft_consensus.cc:348] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.826174 24704 raft_consensus.cc:374] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:10.826285 24704 raft_consensus.cc:729] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c2aa31b4e747eb8e628d1c2b0e2946, State: Initialized, Role: FOLLOWER
I20260806 19:42:10.826647 24704 consensus_queue.cc:260] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:10.828197 24704 ts_tablet_manager.cc:1428] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 19:42:11.038480 24669 raft_consensus.cc:482] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:11.038754 24669 raft_consensus.cc:504] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:11.039763 24669 leader_election.cc:288] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8492820db869485582cc539ba164fac6 (127.21.202.65:37029), 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313)
I20260806 19:42:11.040397 24389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd45e4cabb8d4496a1abc743e1320573" candidate_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" is_pre_election: true
I20260806 19:42:11.040398 24247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd45e4cabb8d4496a1abc743e1320573" candidate_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8492820db869485582cc539ba164fac6" is_pre_election: true
I20260806 19:42:11.040668 24389 raft_consensus.cc:2454] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1c2aa31b4e747eb8e628d1c2b0e2946 in term 0.
I20260806 19:42:11.040869 24247 raft_consensus.cc:2454] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1c2aa31b4e747eb8e628d1c2b0e2946 in term 0.
I20260806 19:42:11.041275 24464 leader_election.cc:302] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ef7444fa3f4a869f9abeefca5b0a5c, d1c2aa31b4e747eb8e628d1c2b0e2946; no voters:
I20260806 19:42:11.041621 24669 raft_consensus.cc:2781] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:42:11.041741 24669 raft_consensus.cc:482] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:11.041886 24669 raft_consensus.cc:3037] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:11.044292 24669 raft_consensus.cc:504] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:11.045010 24669 leader_election.cc:288] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [CANDIDATE]: Term 1 election: Requested vote from peers 8492820db869485582cc539ba164fac6 (127.21.202.65:37029), 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313)
I20260806 19:42:11.045553 24247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd45e4cabb8d4496a1abc743e1320573" candidate_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8492820db869485582cc539ba164fac6"
I20260806 19:42:11.045553 24389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd45e4cabb8d4496a1abc743e1320573" candidate_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c"
I20260806 19:42:11.045782 24247 raft_consensus.cc:3037] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:11.045794 24389 raft_consensus.cc:3037] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:11.048099 24389 raft_consensus.cc:2454] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c2aa31b4e747eb8e628d1c2b0e2946 in term 1.
I20260806 19:42:11.048391 24247 raft_consensus.cc:2454] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c2aa31b4e747eb8e628d1c2b0e2946 in term 1.
I20260806 19:42:11.048678 24464 leader_election.cc:302] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ef7444fa3f4a869f9abeefca5b0a5c, d1c2aa31b4e747eb8e628d1c2b0e2946; no voters:
I20260806 19:42:11.049069 24669 raft_consensus.cc:2781] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:11.049261 24669 raft_consensus.cc:686] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 LEADER]: Becoming Leader. State: Replica: d1c2aa31b4e747eb8e628d1c2b0e2946, State: Running, Role: LEADER
I20260806 19:42:11.049608 24669 consensus_queue.cc:237] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } }
I20260806 19:42:11.052598 24080 catalog_manager.cc:5495] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1c2aa31b4e747eb8e628d1c2b0e2946 (127.21.202.67). New cstate: current_term: 1 leader_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1c2aa31b4e747eb8e628d1c2b0e2946" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43657 } health_report { overall_health: HEALTHY } } }
I20260806 19:42:11.090281 24389 raft_consensus.cc:1261] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Refusing update from remote peer d1c2aa31b4e747eb8e628d1c2b0e2946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:11.090432 24247 raft_consensus.cc:1261] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Refusing update from remote peer d1c2aa31b4e747eb8e628d1c2b0e2946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:11.090931 24669 consensus_queue.cc:1035] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ef7444fa3f4a869f9abeefca5b0a5c" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 34313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:11.091238 24683 consensus_queue.cc:1035] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8492820db869485582cc539ba164fac6" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 19:42:11.115422 24673 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 19:42:12.713646 24079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:12.713891 24079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:12.719058 24079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0439f0f563e3412f8173359deac61e6e: Sending DeleteTablet for 3 replicas of tablet dd45e4cabb8d4496a1abc743e1320573
I20260806 19:42:12.720230 24370 tablet_service.cc:1514] Processing DeleteTablet for tablet dd45e4cabb8d4496a1abc743e1320573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:12 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:40934
I20260806 19:42:12.720577 24226 tablet_service.cc:1514] Processing DeleteTablet for tablet dd45e4cabb8d4496a1abc743e1320573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:12 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:36622
I20260806 19:42:12.720973 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 24161
I20260806 19:42:12.721266 24510 tablet_service.cc:1514] Processing DeleteTablet for tablet dd45e4cabb8d4496a1abc743e1320573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:12 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:48420
I20260806 19:42:12.723340 24746 tablet_replica.cc:330] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c: stopping tablet replica
I20260806 19:42:12.723588 24746 raft_consensus.cc:2229] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:12.723867 24746 raft_consensus.cc:2258] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:12.725035 24746 ts_tablet_manager.cc:1905] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:12.732199 24746 ts_tablet_manager.cc:1918] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 19:42:12.732445 24746 log.cc:1196] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/dd45e4cabb8d4496a1abc743e1320573
I20260806 19:42:12.732997 24746 ts_tablet_manager.cc:1939] T dd45e4cabb8d4496a1abc743e1320573 P 30ef7444fa3f4a869f9abeefca5b0a5c: Deleting consensus metadata
I20260806 19:42:12.734617 24069 catalog_manager.cc:4841] TS 30ef7444fa3f4a869f9abeefca5b0a5c (127.21.202.66:34313): tablet dd45e4cabb8d4496a1abc743e1320573 (table test-table [id=b9306e63fae446839f793272043344a2]) successfully deleted
I20260806 19:42:12.746210 24747 tablet_replica.cc:330] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6: stopping tablet replica
I20260806 19:42:12.746661 24747 raft_consensus.cc:2229] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:12.746920 24748 tablet_replica.cc:330] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946: stopping tablet replica
I20260806 19:42:12.746955 24747 raft_consensus.cc:2258] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:12.747170 24748 raft_consensus.cc:2229] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:42:12.747648 24748 raft_consensus.cc:2258] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:12.748106 24747 ts_tablet_manager.cc:1905] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:12.748749 24748 ts_tablet_manager.cc:1905] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:12.754420 24747 ts_tablet_manager.cc:1918] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 19:42:12.754546 24747 log.cc:1196] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/dd45e4cabb8d4496a1abc743e1320573
I20260806 19:42:12.754635 24748 ts_tablet_manager.cc:1918] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 19:42:12.754777 24748 log.cc:1196] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/dd45e4cabb8d4496a1abc743e1320573
I20260806 19:42:12.755079 24747 ts_tablet_manager.cc:1939] T dd45e4cabb8d4496a1abc743e1320573 P 8492820db869485582cc539ba164fac6: Deleting consensus metadata
I20260806 19:42:12.755255 24748 ts_tablet_manager.cc:1939] T dd45e4cabb8d4496a1abc743e1320573 P d1c2aa31b4e747eb8e628d1c2b0e2946: Deleting consensus metadata
I20260806 19:42:12.756182 24068 catalog_manager.cc:4841] TS 8492820db869485582cc539ba164fac6 (127.21.202.65:37029): tablet dd45e4cabb8d4496a1abc743e1320573 (table test-table [id=b9306e63fae446839f793272043344a2]) successfully deleted
I20260806 19:42:12.756304 24066 catalog_manager.cc:4841] TS d1c2aa31b4e747eb8e628d1c2b0e2946 (127.21.202.67:43657): tablet dd45e4cabb8d4496a1abc743e1320573 (table test-table [id=b9306e63fae446839f793272043344a2]) successfully deleted
Aug 06 19:42:12 dist-test-slave-77v9 krb5kdc[24037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045330, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:42:12.767158 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:12.722492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34202 (local address 127.21.202.65:37029)
0806 19:42:12.744620 (+ 22128us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:12.744631 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:12.745440 (+ 809us) server_negotiation.cc:407] Connection header received
0806 19:42:12.745718 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:12.745722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:12.745816 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:12.746046 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:12.747913 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:12.749085 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:12.751005 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:12.751735 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:12.751862 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:12.761061 (+ 9199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:12.761094 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:12.761115 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:12.761171 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:12.764068 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:12.764732 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:12.764740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:12.764742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:12.764982 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:12.765572 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:12.765576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:12.765578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:12.765990 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:12.766240 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:12.766540 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 19:42:12.766720 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21849,"thread_start_us":22175,"threads_started":1}
W20260806 19:42:12.767825 24294 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34202
W20260806 19:42:12.768548 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 24161: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:42:12.768903 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 24161
I20260806 19:42:12.801347 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 24303
Aug 06 19:42:12 dist-test-slave-77v9 krb5kdc[24037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045330, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:42:12.819664 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:12.802467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36226 (local address 127.21.202.66:34313)
0806 19:42:12.803144 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:12.803150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:12.803168 (+ 18us) server_negotiation.cc:407] Connection header received
0806 19:42:12.803228 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:12.803233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:12.803300 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:12.803443 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:12.804979 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:12.805800 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:12.807253 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:12.807729 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:12.807891 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:12.812760 (+ 4869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:12.812777 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:12.812794 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:12.812910 (+ 116us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:12.816248 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:12.817180 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:12.817189 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:12.817191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:12.817360 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:12.817780 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:12.817786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:12.817790 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:12.818197 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:12.818627 (+ 430us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:12.818989 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 19:42:12.819253 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":357,"threads_started":1}
W20260806 19:42:12.820372 24436 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36226
W20260806 19:42:12.821148 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 24303: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:42:12.821307 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 24303
I20260806 19:42:12.841343 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 24445
Aug 06 19:42:12 dist-test-slave-77v9 krb5kdc[24037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045330, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:42:12.859453 24752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:12.842218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60448 (local address 127.21.202.67:43657)
0806 19:42:12.842786 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:12.842790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:12.842807 (+ 17us) server_negotiation.cc:407] Connection header received
0806 19:42:12.842864 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:12.842868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:12.842934 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:12.843027 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:12.844529 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:12.845535 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:12.847059 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:12.847533 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:12.847647 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:12.852529 (+ 4882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:12.852546 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:12.852564 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:12.852605 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:12.856170 (+ 3565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:12.857232 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:12.857240 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:12.857242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:12.857399 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:12.857821 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:12.857831 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:12.857833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:12.858197 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:12.858513 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:12.858825 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 19:42:12.859076 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":301,"threads_started":1}
W20260806 19:42:12.860222 24576 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60448
W20260806 19:42:12.861274 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 24445: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:42:12.861459 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 24445
I20260806 19:42:12.888139 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 24050
I20260806 19:42:12.889483 24124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:13.013541 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 24050
2026-08-06T19:42:13Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8789 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:42:13 dist-test-slave-77v9 krb5kdc[24756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:42:13 dist-test-slave-77v9 krb5kdc[24756](info): set up 2 sockets
Aug 06 19:42:13 dist-test-slave-77v9 krb5kdc[24756](info): commencing operation
krb5kdc: starting...
W20260806 19:42:14.190975 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:42:14 dist-test-slave-77v9 krb5kdc[24756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:42:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:42:14.437467 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:45733
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:39487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:45733
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:14.799499 24769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:14.799827 24769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:14.799927 24769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:14.807549 24769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:14.807691 24769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:14.807749 24769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:14.807808 24769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:14.807862 24769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:42:14.819159 24769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:39487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:45733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:45733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:14.820688 24769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:14.822186 24769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:14.835907 24775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:14.835963 24777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:14.835963 24774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:14.837499 24769 server_base.cc:1034] running on GCE node
I20260806 19:42:14.838626 24769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:14.840335 24769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:14.841620 24769 hybrid_clock.cc:648] HybridClock initialized: now 1786045334841580 us; error 75 us; skew 500 ppm
I20260806 19:42:14.842073 24769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:14 dist-test-slave-77v9 krb5kdc[24756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045334, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:14.845570 24769 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:42:14.847278 24769 webserver.cc:460] Webserver started at http://127.21.202.126:44613/ using document root <none> and password file <none>
I20260806 19:42:14.848035 24769 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:14.848191 24769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:14.848625 24769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:14.852290 24769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "31703d24f9ac49ad883ecf2b6b5c2a5f"
format_stamp: "Formatted at 2026-08-06 19:42:14 on dist-test-slave-77v9"
server_key: "decef40440146c7131c7c939b9de3288"
server_key_iv: "9d3031b6e961e1695133e3c8ce61c343"
server_key_version: "encryptionkey@0"
I20260806 19:42:14.853281 24769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "31703d24f9ac49ad883ecf2b6b5c2a5f"
format_stamp: "Formatted at 2026-08-06 19:42:14 on dist-test-slave-77v9"
server_key: "decef40440146c7131c7c939b9de3288"
server_key_iv: "9d3031b6e961e1695133e3c8ce61c343"
server_key_version: "encryptionkey@0"
I20260806 19:42:14.859586 24769 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260806 19:42:14.863634 24784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:14.865693 24769 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 19:42:14.865914 24769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "31703d24f9ac49ad883ecf2b6b5c2a5f"
format_stamp: "Formatted at 2026-08-06 19:42:14 on dist-test-slave-77v9"
server_key: "decef40440146c7131c7c939b9de3288"
server_key_iv: "9d3031b6e961e1695133e3c8ce61c343"
server_key_version: "encryptionkey@0"
I20260806 19:42:14.866159 24769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:14.885313 24769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:14.890317 24769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:14.890708 24769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:14.912993 24769 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:45733
I20260806 19:42:14.913094 24837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:45733 every 8 connection(s)
I20260806 19:42:14.914587 24769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:14.918485 24838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:14.922214 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 24769
I20260806 19:42:14.922479 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:14.923295 22313 external_mini_cluster.cc:1428] Setting key f4e4de2e6a3e465b1bede31393f418a2
I20260806 19:42:14.933470 24838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f: Bootstrap starting.
I20260806 19:42:14.937902 24838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:14.940013 24838 log.cc:824] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f: Log is configured to *not* fsync() on all Append() calls
Aug 06 19:42:14 dist-test-slave-77v9 krb5kdc[24756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:14.943367 24838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f: No bootstrap required, opened a new log
I20260806 19:42:14.950052 24841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:14.925502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43992 (local address 127.21.202.126:45733)
0806 19:42:14.926380 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:14.926393 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:14.926433 (+ 40us) server_negotiation.cc:407] Connection header received
0806 19:42:14.928188 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:14.928212 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:14.928632 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:14.929048 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:14.931146 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:14.932582 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:14.934684 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:14.935508 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:14.935664 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:14.942166 (+ 6502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:14.942175 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:14.942191 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:14.942228 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:14.945657 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:14.946428 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:14.946436 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:14.946443 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:14.946635 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:14.947050 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:14.947054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:14.947056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:14.947630 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:14.947999 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:14.948436 (+ 437us) server_negotiation.cc:299] Negotiation successful
0806 19:42:14.948962 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":437,"threads_started":1}
I20260806 19:42:14.951555 24838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31703d24f9ac49ad883ecf2b6b5c2a5f" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45733 } }
I20260806 19:42:14.951988 24838 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:14.952114 24838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31703d24f9ac49ad883ecf2b6b5c2a5f, State: Initialized, Role: FOLLOWER
I20260806 19:42:14.952744 24838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31703d24f9ac49ad883ecf2b6b5c2a5f" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45733 } }
I20260806 19:42:14.953066 24838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:14.953200 24838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:14.953361 24838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:14.955682 24838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31703d24f9ac49ad883ecf2b6b5c2a5f" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45733 } }
I20260806 19:42:14.956287 24838 leader_election.cc:302] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31703d24f9ac49ad883ecf2b6b5c2a5f; no voters:
I20260806 19:42:14.956992 24838 leader_election.cc:288] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:14.957239 24843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:14.957983 24843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [term 1 LEADER]: Becoming Leader. State: Replica: 31703d24f9ac49ad883ecf2b6b5c2a5f, State: Running, Role: LEADER
I20260806 19:42:14.958534 24843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31703d24f9ac49ad883ecf2b6b5c2a5f" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45733 } }
I20260806 19:42:14.959108 24838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:14.961753 24846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31703d24f9ac49ad883ecf2b6b5c2a5f. Latest consensus state: current_term: 1 leader_uuid: "31703d24f9ac49ad883ecf2b6b5c2a5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31703d24f9ac49ad883ecf2b6b5c2a5f" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45733 } } }
I20260806 19:42:14.961717 24845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31703d24f9ac49ad883ecf2b6b5c2a5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31703d24f9ac49ad883ecf2b6b5c2a5f" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45733 } } }
I20260806 19:42:14.962252 24845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:14.962225 24846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:14.967077 24849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:14.974303 24849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:14.983893 24849 catalog_manager.cc:1347] Generated new cluster ID: 42fe34bfa7e84b48bd6ef2b3944e8c46
I20260806 19:42:14.984027 24849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:15.004544 24849 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:15.006062 24849 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:15.015520 24849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31703d24f9ac49ad883ecf2b6b5c2a5f: Generated new TSK 0
I20260806 19:42:15.016476 24849 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:42:15.090445 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:45733
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:39487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:15.469077 24867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:15.469383 24867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:15.469482 24867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:15.477686 24867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:15.477833 24867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:15.477969 24867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:42:15.488801 24867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:39487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:45733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:15.490176 24867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:15.491561 24867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:15.505183 24872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:15.505149 24875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:15.505239 24873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:15.511615 24867 server_base.cc:1034] running on GCE node
I20260806 19:42:15.512296 24867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:15.513463 24867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:15.514746 24867 hybrid_clock.cc:648] HybridClock initialized: now 1786045335514699 us; error 39 us; skew 500 ppm
I20260806 19:42:15.515146 24867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:15 dist-test-slave-77v9 krb5kdc[24756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045335, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:15.519562 24867 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:42:15.521548 24867 webserver.cc:460] Webserver started at http://127.21.202.65:41613/ using document root <none> and password file <none>
I20260806 19:42:15.522382 24867 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:15.522548 24867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:15.522898 24867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:15.526367 24867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "86f0b061f6f2487b8c9b60e508f05e3f"
format_stamp: "Formatted at 2026-08-06 19:42:15 on dist-test-slave-77v9"
server_key: "df179f9ec0c69122415dbf2b7dee009d"
server_key_iv: "7ca87e557403f6eea9ee37c0e883b732"
server_key_version: "encryptionkey@0"
I20260806 19:42:15.527426 24867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "86f0b061f6f2487b8c9b60e508f05e3f"
format_stamp: "Formatted at 2026-08-06 19:42:15 on dist-test-slave-77v9"
server_key: "df179f9ec0c69122415dbf2b7dee009d"
server_key_iv: "7ca87e557403f6eea9ee37c0e883b732"
server_key_version: "encryptionkey@0"
I20260806 19:42:15.533309 24867 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 19:42:15.537147 24882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:15.538684 24867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:42:15.538829 24867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "86f0b061f6f2487b8c9b60e508f05e3f"
format_stamp: "Formatted at 2026-08-06 19:42:15 on dist-test-slave-77v9"
server_key: "df179f9ec0c69122415dbf2b7dee009d"
server_key_iv: "7ca87e557403f6eea9ee37c0e883b732"
server_key_version: "encryptionkey@0"
I20260806 19:42:15.539029 24867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:15.576264 24867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:15.581067 24867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:15.581338 24867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:15.582369 24867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:15.583827 24867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:15.583914 24867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:15.584014 24867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:15.584064 24867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:15.621820 24867 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:32783
I20260806 19:42:15.621876 24997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:32783 every 8 connection(s)
I20260806 19:42:15.623262 24867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:42:15.629242 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 24867
I20260806 19:42:15.629491 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:42:15.630308 22313 external_mini_cluster.cc:1428] Setting key f53db5b4eaecbb086b77950157c42ab7
Aug 06 19:42:15 dist-test-slave-77v9 krb5kdc[24756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045335, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:15.648415 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:15.626226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:46835 (local address 127.21.202.126:45733)
0806 19:42:15.626738 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:15.626743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:15.627862 (+ 1119us) server_negotiation.cc:407] Connection header received
0806 19:42:15.630560 (+ 2698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:15.630565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:15.630646 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:15.630796 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:15.633021 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:15.633777 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:15.635393 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:15.635801 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:15.635944 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:15.640284 (+ 4340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:15.640294 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:15.640303 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:15.640359 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:15.643724 (+ 3365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:15.644932 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:15.644940 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:15.644943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:15.645157 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:15.646021 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:15.646025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:15.646027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:15.646348 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:15.646496 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:15.647630 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0806 19:42:15.648023 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":285,"threads_started":1}
I20260806 19:42:15.650017 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:15.626953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45733 (local address 127.21.202.65:46835)
0806 19:42:15.627690 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:15.627729 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:15.630148 (+ 2419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:15.631158 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:15.631170 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:15.631955 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:15.632791 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:15.632870 (+ 79us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:15.633954 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:15.633959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:15.635229 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:15.635243 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:15.636070 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:15.636078 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:15.636310 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:15.637062 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:15.637091 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:15.639958 (+ 2867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:15.643907 (+ 3949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:15.643920 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:15.643942 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:15.644648 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:15.645306 (+ 658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:15.645313 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:15.645319 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:15.645697 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:15.646512 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:15.646523 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:15.647088 (+ 565us) client_negotiation.cc:769] Sending connection context
0806 19:42:15.647461 (+ 373us) client_negotiation.cc:240] Negotiation successful
0806 19:42:15.648126 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":320,"threads_started":1}
I20260806 19:42:15.652726 24998 heartbeater.cc:347] Connected to a master server at 127.21.202.126:45733
I20260806 19:42:15.653144 24998 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:15.654192 24998 heartbeater.cc:511] Master 127.21.202.126:45733 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
I20260806 19:42:15.657073 24802 ts_manager.cc:194] Registered new tserver with Master: 86f0b061f6f2487b8c9b60e508f05e3f (127.21.202.65:32783)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
I20260806 19:42:15.658675 24802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:46835
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:42:15.688179 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:45733
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:39487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:16.063164 25005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:16.063591 25005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:16.063787 25005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:16.071890 25005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:16.072105 25005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:16.072338 25005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:42:16.083583 25005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:39487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:45733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:16.085177 25005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:16.086743 25005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:16.101773 25011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:16.101797 25013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:16.101989 25010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:16.103139 25005 server_base.cc:1034] running on GCE node
I20260806 19:42:16.103865 25005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:16.105068 25005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:16.106352 25005 hybrid_clock.cc:648] HybridClock initialized: now 1786045336106296 us; error 34 us; skew 500 ppm
I20260806 19:42:16.106724 25005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:16 dist-test-slave-77v9 krb5kdc[24756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045336, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:16.110586 25005 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:42:16.112412 25005 webserver.cc:460] Webserver started at http://127.21.202.66:34849/ using document root <none> and password file <none>
I20260806 19:42:16.113291 25005 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:16.113456 25005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:16.113812 25005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:16.117270 25005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "70c6778e03ed462a83a239dd3ffff55f"
format_stamp: "Formatted at 2026-08-06 19:42:16 on dist-test-slave-77v9"
server_key: "ade69cba27212d69fb4185f8a17cf56e"
server_key_iv: "7ed2a0c26ba5799bc01a32d08b106a4c"
server_key_version: "encryptionkey@0"
I20260806 19:42:16.118244 25005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "70c6778e03ed462a83a239dd3ffff55f"
format_stamp: "Formatted at 2026-08-06 19:42:16 on dist-test-slave-77v9"
server_key: "ade69cba27212d69fb4185f8a17cf56e"
server_key_iv: "7ed2a0c26ba5799bc01a32d08b106a4c"
server_key_version: "encryptionkey@0"
I20260806 19:42:16.124374 25005 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 19:42:16.128407 25020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:16.130153 25005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:16.130373 25005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "70c6778e03ed462a83a239dd3ffff55f"
format_stamp: "Formatted at 2026-08-06 19:42:16 on dist-test-slave-77v9"
server_key: "ade69cba27212d69fb4185f8a17cf56e"
server_key_iv: "7ed2a0c26ba5799bc01a32d08b106a4c"
server_key_version: "encryptionkey@0"
I20260806 19:42:16.130625 25005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:16.145437 25005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:16.150566 25005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:16.150918 25005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:16.152005 25005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:16.153528 25005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:16.153680 25005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 19:42:16.153810 25005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:16.153852 25005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:16.193563 25005 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:44807
I20260806 19:42:16.193615 25133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:44807 every 8 connection(s)
I20260806 19:42:16.195006 25005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:42:16.203402 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 25005
I20260806 19:42:16.203619 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:42:16.204205 22313 external_mini_cluster.cc:1428] Setting key 87ccb6900d0b0743d16bafd28b56df44
Aug 06 19:42:16 dist-test-slave-77v9 krb5kdc[24756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045336, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:16.221047 25137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:16.198202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:48979 (local address 127.21.202.126:45733)
0806 19:42:16.198757 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:16.198766 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:16.199939 (+ 1173us) server_negotiation.cc:407] Connection header received
0806 19:42:16.201959 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:16.201968 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:16.202067 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:16.202180 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:16.203979 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:16.205051 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:16.207733 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:16.208681 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:16.208916 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:16.212965 (+ 4049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:16.212975 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:16.212984 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:16.213038 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:16.216483 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:16.217605 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:16.217611 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:16.217613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:16.217821 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:16.218450 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:16.218455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:16.218457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:16.218836 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:16.218979 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:16.220323 (+ 1344us) server_negotiation.cc:299] Negotiation successful
0806 19:42:16.220564 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":318,"threads_started":1}
I20260806 19:42:16.222465 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:16.198901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45733 (local address 127.21.202.66:48979)
0806 19:42:16.199739 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:16.199799 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:16.201648 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:16.202395 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:16.202403 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:16.203079 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:16.203791 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:16.203812 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:16.205334 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:16.205425 (+ 91us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:16.207532 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:16.207549 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:16.209051 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:16.209057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:16.209278 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:16.210240 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:16.210264 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:16.212621 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:16.216683 (+ 4062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:16.216696 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:16.216719 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:16.217452 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:16.217960 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:16.217969 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:16.217975 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:16.218289 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:16.219046 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:16.219058 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:16.219596 (+ 538us) client_negotiation.cc:769] Sending connection context
0806 19:42:16.220056 (+ 460us) client_negotiation.cc:240] Negotiation successful
0806 19:42:16.220875 (+ 819us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":374,"threads_started":1}
I20260806 19:42:16.224440 25134 heartbeater.cc:347] Connected to a master server at 127.21.202.126:45733
I20260806 19:42:16.224853 25134 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:16.225844 25134 heartbeater.cc:511] Master 127.21.202.126:45733 requested a full tablet report, sending...
I20260806 19:42:16.227811 24802 ts_manager.cc:194] Registered new tserver with Master: 70c6778e03ed462a83a239dd3ffff55f (127.21.202.66:44807)
I20260806 19:42:16.229111 24802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:48979
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
I20260806 19:42:16.662981 24998 heartbeater.cc:503] Master 127.21.202.126:45733 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
W20260806 19:42:17.206610 22313 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.202.67: real 0.999s user 0.003s sys 0.013s
I20260806 19:42:17.207639 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:45733
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:39487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:42:17.232236 25134 heartbeater.cc:503] Master 127.21.202.126:45733 was elected leader, sending a full tablet report...
W20260806 19:42:17.598268 25141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:17.598663 25141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:17.598917 25141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:17.606689 25141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:17.606875 25141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:17.607081 25141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:42:17.617561 25141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:39487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:45733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:17.619036 25141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:17.620471 25141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:17.634804 25149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:17.635051 25146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:17.635031 25147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:17.636255 25141 server_base.cc:1034] running on GCE node
I20260806 19:42:17.637033 25141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:17.638164 25141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:17.639470 25141 hybrid_clock.cc:648] HybridClock initialized: now 1786045337639481 us; error 175 us; skew 500 ppm
I20260806 19:42:17.639991 25141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:17 dist-test-slave-77v9 krb5kdc[24756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045337, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:17.644006 25141 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:42:17.645828 25141 webserver.cc:460] Webserver started at http://127.21.202.67:37259/ using document root <none> and password file <none>
I20260806 19:42:17.646648 25141 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:17.646816 25141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:17.647184 25141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:17.650746 25141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "c27ef2b5219f4e9c955314dc1c495749"
format_stamp: "Formatted at 2026-08-06 19:42:17 on dist-test-slave-77v9"
server_key: "6528a68a0d6322662929a6b659483763"
server_key_iv: "9fdfd232b5d71e4a65df22c28fce127e"
server_key_version: "encryptionkey@0"
I20260806 19:42:17.651731 25141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "c27ef2b5219f4e9c955314dc1c495749"
format_stamp: "Formatted at 2026-08-06 19:42:17 on dist-test-slave-77v9"
server_key: "6528a68a0d6322662929a6b659483763"
server_key_iv: "9fdfd232b5d71e4a65df22c28fce127e"
server_key_version: "encryptionkey@0"
I20260806 19:42:17.658876 25141 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 19:42:17.663275 25156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:17.665146 25141 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:42:17.665343 25141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "c27ef2b5219f4e9c955314dc1c495749"
format_stamp: "Formatted at 2026-08-06 19:42:17 on dist-test-slave-77v9"
server_key: "6528a68a0d6322662929a6b659483763"
server_key_iv: "9fdfd232b5d71e4a65df22c28fce127e"
server_key_version: "encryptionkey@0"
I20260806 19:42:17.665620 25141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:17.689365 25141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:17.694743 25141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:17.695071 25141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:17.696205 25141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:17.697854 25141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:17.697971 25141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:17.698154 25141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:17.698225 25141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:17.739362 25141 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:41311
I20260806 19:42:17.739428 25272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:41311 every 8 connection(s)
I20260806 19:42:17.740761 25141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:42:17.745075 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 25141
I20260806 19:42:17.745291 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:42:17.746070 22313 external_mini_cluster.cc:1428] Setting key 4f028ca02749084c03038c9c73621d49
Aug 06 19:42:17 dist-test-slave-77v9 krb5kdc[24756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045337, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:17.771651 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:17.743778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:33255 (local address 127.21.202.126:45733)
0806 19:42:17.744329 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:17.744334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:17.745578 (+ 1244us) server_negotiation.cc:407] Connection header received
0806 19:42:17.747966 (+ 2388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:17.747974 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:17.748096 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:17.748254 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:17.750747 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:17.751636 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:17.753607 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:17.754116 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:17.754228 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:17.758372 (+ 4144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:17.758379 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:17.758383 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:17.758418 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:17.767276 (+ 8858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:17.768321 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:17.768326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:17.768328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:17.768477 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:17.769195 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:17.769200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:17.769202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:17.769547 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:17.769690 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:17.770773 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0806 19:42:17.771215 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":288,"threads_started":1}
I20260806 19:42:17.772609 25276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:17.744433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45733 (local address 127.21.202.67:33255)
0806 19:42:17.745378 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:17.745438 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:17.747663 (+ 2225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:17.748683 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:17.748694 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:17.749750 (+ 1056us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:17.750538 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:17.750558 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:17.751799 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:17.751805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:17.753316 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:17.753330 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:17.754381 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:17.754387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:17.754620 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:17.755347 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:17.755373 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:17.758080 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:17.767470 (+ 9390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:17.767480 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:17.767497 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:17.768163 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:17.768633 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:17.768638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:17.768641 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:17.769031 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:17.769685 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:17.769692 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:17.770276 (+ 584us) client_negotiation.cc:769] Sending connection context
0806 19:42:17.770648 (+ 372us) client_negotiation.cc:240] Negotiation successful
0806 19:42:17.771250 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":424,"threads_started":1}
I20260806 19:42:17.774531 25273 heartbeater.cc:347] Connected to a master server at 127.21.202.126:45733
I20260806 19:42:17.774900 25273 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:17.775698 25273 heartbeater.cc:511] Master 127.21.202.126:45733 requested a full tablet report, sending...
I20260806 19:42:17.777397 24801 ts_manager.cc:194] Registered new tserver with Master: c27ef2b5219f4e9c955314dc1c495749 (127.21.202.67:41311)
I20260806 19:42:17.778298 24801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:33255
I20260806 19:42:17.788391 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 19:42:17.827315 25275 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:42:17.824018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44000 (local address 127.21.202.126:45733)
0806 19:42:17.824313 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:17.824321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:17.824684 (+ 363us) server_negotiation.cc:407] Connection header received
0806 19:42:17.824932 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:17.824937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:17.825010 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:17.825111 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:17.826970 (+ 1859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44000: BlockingRecv error: recv got EOF from 127.0.0.1:44000 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 19:42:17.831786 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 24867
W20260806 19:42:17.835027 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 24867: Not authorized: Client connection negotiation failed: client connection to 127.21.202.65:32783: server requires authentication, but client does not have Kerberos credentials available
I20260806 19:42:17.835295 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 24867
I20260806 19:42:17.852875 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 25005
W20260806 19:42:17.856052 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 25005: Not authorized: Client connection negotiation failed: client connection to 127.21.202.66:44807: server requires authentication, but client does not have Kerberos credentials available
I20260806 19:42:17.856241 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 25005
I20260806 19:42:17.875656 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 25141
W20260806 19:42:17.879248 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 25141: Not authorized: Client connection negotiation failed: client connection to 127.21.202.67:41311: server requires authentication, but client does not have Kerberos credentials available
I20260806 19:42:17.879456 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 25141
I20260806 19:42:17.897967 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 24769
I20260806 19:42:17.899439 24833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:18.009509 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 24769
2026-08-06T19:42:18Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4986 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:42:18 dist-test-slave-77v9 krb5kdc[25292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:42:18 dist-test-slave-77v9 krb5kdc[25292](info): set up 2 sockets
Aug 06 19:42:18 dist-test-slave-77v9 krb5kdc[25292](info): commencing operation
krb5kdc: starting...
W20260806 19:42:19.178000 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:42:19 dist-test-slave-77v9 krb5kdc[25292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:42:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:42:19.413651 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:36573
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:36573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:19.754649 25305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:19.754952 25305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:19.755111 25305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:19.763382 25305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:19.763511 25305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:19.763633 25305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:19.763741 25305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:19.763818 25305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:42:19.775631 25305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:36573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:36573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:19.777190 25305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:19.778666 25305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:19.791841 25310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:19.792035 25311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:19.792073 25313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:19.793761 25305 server_base.cc:1034] running on GCE node
I20260806 19:42:19.794625 25305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:19.796195 25305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:19.797487 25305 hybrid_clock.cc:648] HybridClock initialized: now 1786045339797416 us; error 55 us; skew 500 ppm
I20260806 19:42:19.797940 25305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:19 dist-test-slave-77v9 krb5kdc[25292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045339, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:19.801571 25305 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:42:19.803346 25305 webserver.cc:460] Webserver started at http://127.21.202.126:34761/ using document root <none> and password file <none>
I20260806 19:42:19.804061 25305 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:19.804214 25305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:19.804553 25305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:19.807785 25305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "61eb671edd57451eb3b875fb695819e1"
format_stamp: "Formatted at 2026-08-06 19:42:19 on dist-test-slave-77v9"
server_key: "e70e8d165fd961c0e400d057c2b30dc8"
server_key_iv: "dfa0db7cc00bfe7004a61ad7683b2b3d"
server_key_version: "encryptionkey@0"
I20260806 19:42:19.808601 25305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "61eb671edd57451eb3b875fb695819e1"
format_stamp: "Formatted at 2026-08-06 19:42:19 on dist-test-slave-77v9"
server_key: "e70e8d165fd961c0e400d057c2b30dc8"
server_key_iv: "dfa0db7cc00bfe7004a61ad7683b2b3d"
server_key_version: "encryptionkey@0"
I20260806 19:42:19.813908 25305 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 19:42:19.817461 25320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:19.818876 25305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:42:19.819077 25305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "61eb671edd57451eb3b875fb695819e1"
format_stamp: "Formatted at 2026-08-06 19:42:19 on dist-test-slave-77v9"
server_key: "e70e8d165fd961c0e400d057c2b30dc8"
server_key_iv: "dfa0db7cc00bfe7004a61ad7683b2b3d"
server_key_version: "encryptionkey@0"
I20260806 19:42:19.819294 25305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:19.838918 25305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:19.843428 25305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:19.843888 25305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:19.865314 25305 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:36573
I20260806 19:42:19.865319 25373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:36573 every 8 connection(s)
I20260806 19:42:19.867125 25305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:19.867847 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 25305
I20260806 19:42:19.868176 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:19.869125 22313 external_mini_cluster.cc:1428] Setting key cd24a73c75f34beace2afa7de89927e2
I20260806 19:42:19.872710 25374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 19:42:19 dist-test-slave-77v9 krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:19.888902 25374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1: Bootstrap starting.
I20260806 19:42:19.892748 25374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:19.894544 25374 log.cc:824] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:19.898438 25374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1: No bootstrap required, opened a new log
I20260806 19:42:19.898355 25377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:19.871709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.21.202.126:36573)
0806 19:42:19.872781 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:19.872801 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:19.872912 (+ 111us) server_negotiation.cc:407] Connection header received
0806 19:42:19.875754 (+ 2842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:19.875786 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:19.876364 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:19.876965 (+ 601us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:19.878723 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:19.881135 (+ 2412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:19.882575 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:19.883825 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:19.884060 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:19.889148 (+ 5088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:19.889160 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:19.889177 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:19.889219 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:19.893102 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:19.894395 (+ 1293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:19.894402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:19.894410 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:19.894630 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:19.895119 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:19.895127 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:19.895131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:19.895759 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:19.896131 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:19.896656 (+ 525us) server_negotiation.cc:299] Negotiation successful
0806 19:42:19.897153 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":384,"threads_started":1}
I20260806 19:42:19.903187 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:42:19.907776 25374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61eb671edd57451eb3b875fb695819e1" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36573 } }
I20260806 19:42:19.908246 25374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:19.908391 25374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61eb671edd57451eb3b875fb695819e1, State: Initialized, Role: FOLLOWER
I20260806 19:42:19.909314 25374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61eb671edd57451eb3b875fb695819e1" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36573 } }
I20260806 19:42:19.909633 25374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:19.909920 25374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:19.910162 25374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:19.912950 25374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61eb671edd57451eb3b875fb695819e1" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36573 } }
I20260806 19:42:19.913731 25374 leader_election.cc:302] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61eb671edd57451eb3b875fb695819e1; no voters:
I20260806 19:42:19.914569 25374 leader_election.cc:288] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:19.914731 25380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:19.916165 25380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [term 1 LEADER]: Becoming Leader. State: Replica: 61eb671edd57451eb3b875fb695819e1, State: Running, Role: LEADER
I20260806 19:42:19.916993 25380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61eb671edd57451eb3b875fb695819e1" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36573 } }
I20260806 19:42:19.917635 25374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:19.920284 25382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61eb671edd57451eb3b875fb695819e1. Latest consensus state: current_term: 1 leader_uuid: "61eb671edd57451eb3b875fb695819e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61eb671edd57451eb3b875fb695819e1" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36573 } } }
I20260806 19:42:19.920797 25382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:19.920471 25381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61eb671edd57451eb3b875fb695819e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61eb671edd57451eb3b875fb695819e1" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36573 } } }
I20260806 19:42:19.921016 25381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:19.929766 25387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:19.937274 25387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:19.948482 25387 catalog_manager.cc:1347] Generated new cluster ID: 62486d27371e482380596a007fc837ad
I20260806 19:42:19.948609 25387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:19.967295 25377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:19.955617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49590 (local address 127.21.202.126:36573)
0806 19:42:19.955837 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:19.955844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:19.956704 (+ 860us) server_negotiation.cc:407] Connection header received
0806 19:42:19.956961 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:19.956967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:19.957047 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:19.957158 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:19.958757 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:19.959905 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:19.962169 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:19.962835 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:19.962979 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:19.964604 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:19.964610 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:19.964615 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:19.965698 (+ 1083us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:19.965758 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:19.965792 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:19.965916 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:19.966115 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 19:42:19.966462 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 19:42:19.972337 25387 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:19.974567 25387 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:19.985989 25387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61eb671edd57451eb3b875fb695819e1: Generated new TSK 0
I20260806 19:42:19.986915 25387 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:42:20.000092 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 25305
I20260806 19:42:20.001539 25368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:20.098667 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 25305
2026-08-06T19:42:20Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2086 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:42:20 dist-test-slave-77v9 krb5kdc[25408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:42:20 dist-test-slave-77v9 krb5kdc[25408](info): set up 2 sockets
Aug 06 19:42:20 dist-test-slave-77v9 krb5kdc[25408](info): commencing operation
krb5kdc: starting...
W20260806 19:42:21.257953 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:42:21 dist-test-slave-77v9 krb5kdc[25408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:42:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:42:21.497195 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:34833
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:46779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:34833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:21.836890 25421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:21.837210 25421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:21.837316 25421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:21.844650 25421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:21.844758 25421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:21.844852 25421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:21.844929 25421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:21.844975 25421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:42:21.855031 25421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:46779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:34833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:34833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:21.856396 25421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:21.857712 25421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:21.869894 25427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:21.869987 25428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:21.869866 25430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:21.870677 25421 server_base.cc:1034] running on GCE node
I20260806 19:42:21.871531 25421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:21.873066 25421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:21.874325 25421 hybrid_clock.cc:648] HybridClock initialized: now 1786045341874268 us; error 53 us; skew 500 ppm
I20260806 19:42:21.874677 25421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:21 dist-test-slave-77v9 krb5kdc[25408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045341, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:21.877905 25421 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:42:21.879460 25421 webserver.cc:460] Webserver started at http://127.21.202.126:34513/ using document root <none> and password file <none>
I20260806 19:42:21.880147 25421 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:21.880295 25421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:21.880617 25421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:21.883662 25421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "5b21b87212f7408ea5cd34ea608c0d49"
format_stamp: "Formatted at 2026-08-06 19:42:21 on dist-test-slave-77v9"
server_key: "8545c39bd90175dcc232734eb756200b"
server_key_iv: "6e603cf5b4ed70b3d9439bd0e976d7e2"
server_key_version: "encryptionkey@0"
I20260806 19:42:21.884454 25421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "5b21b87212f7408ea5cd34ea608c0d49"
format_stamp: "Formatted at 2026-08-06 19:42:21 on dist-test-slave-77v9"
server_key: "8545c39bd90175dcc232734eb756200b"
server_key_iv: "6e603cf5b4ed70b3d9439bd0e976d7e2"
server_key_version: "encryptionkey@0"
I20260806 19:42:21.889827 25421 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 19:42:21.893373 25437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:21.894711 25421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 19:42:21.894902 25421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "5b21b87212f7408ea5cd34ea608c0d49"
format_stamp: "Formatted at 2026-08-06 19:42:21 on dist-test-slave-77v9"
server_key: "8545c39bd90175dcc232734eb756200b"
server_key_iv: "6e603cf5b4ed70b3d9439bd0e976d7e2"
server_key_version: "encryptionkey@0"
I20260806 19:42:21.895115 25421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:21.912348 25421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:21.916508 25421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:21.916862 25421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:21.936520 25421 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:34833
I20260806 19:42:21.936571 25489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:34833 every 8 connection(s)
I20260806 19:42:21.938054 25421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:21.940980 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 25421
I20260806 19:42:21.941250 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:21.942062 22313 external_mini_cluster.cc:1428] Setting key af6fe9b1f32b5ff6e81859649d7c0a21
I20260806 19:42:21.942515 25490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:21.957885 25490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49: Bootstrap starting.
Aug 06 19:42:21 dist-test-slave-77v9 krb5kdc[25408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:21.962118 25490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:21.963538 25490 log.cc:824] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:21.966650 25490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49: No bootstrap required, opened a new log
I20260806 19:42:21.967903 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:21.944149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45474 (local address 127.21.202.126:34833)
0806 19:42:21.945000 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:21.945012 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:21.945087 (+ 75us) server_negotiation.cc:407] Connection header received
0806 19:42:21.947365 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:21.947396 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:21.947803 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:21.948166 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:21.950112 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:21.951449 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:21.953402 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:21.954173 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:21.954318 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:21.959229 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:21.959240 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:21.959261 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:21.959317 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:21.963567 (+ 4250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:21.964357 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:21.964365 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:21.964374 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:21.964554 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:21.965040 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:21.965044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:21.965046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:21.965588 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:21.965923 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:21.966366 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 19:42:21.966874 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":384,"threads_started":1}
I20260806 19:42:21.974898 25490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b21b87212f7408ea5cd34ea608c0d49" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34833 } }
I20260806 19:42:21.975303 25490 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:21.975414 25490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b21b87212f7408ea5cd34ea608c0d49, State: Initialized, Role: FOLLOWER
I20260806 19:42:21.976013 25490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b21b87212f7408ea5cd34ea608c0d49" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34833 } }
I20260806 19:42:21.976243 25490 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:21.976392 25490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:21.976572 25490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:21.978863 25490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b21b87212f7408ea5cd34ea608c0d49" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34833 } }
I20260806 19:42:21.979385 25490 leader_election.cc:302] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b21b87212f7408ea5cd34ea608c0d49; no voters:
I20260806 19:42:21.980062 25490 leader_election.cc:288] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:21.980201 25495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:21.981498 25495 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [term 1 LEADER]: Becoming Leader. State: Replica: 5b21b87212f7408ea5cd34ea608c0d49, State: Running, Role: LEADER
I20260806 19:42:21.982035 25495 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b21b87212f7408ea5cd34ea608c0d49" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34833 } }
I20260806 19:42:21.982386 25490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:21.985004 25497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b21b87212f7408ea5cd34ea608c0d49. Latest consensus state: current_term: 1 leader_uuid: "5b21b87212f7408ea5cd34ea608c0d49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b21b87212f7408ea5cd34ea608c0d49" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34833 } } }
I20260806 19:42:21.985378 25497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:21.985903 25496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b21b87212f7408ea5cd34ea608c0d49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b21b87212f7408ea5cd34ea608c0d49" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34833 } } }
I20260806 19:42:21.986343 25496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:21.992980 25504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:21.998605 25504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:22.007848 25504 catalog_manager.cc:1347] Generated new cluster ID: dd1d2001bf154ac0ae1b855776cbc4c0
I20260806 19:42:22.007998 25504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:22.014937 25504 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:22.016374 25504 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:22.025435 25504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b21b87212f7408ea5cd34ea608c0d49: Generated new TSK 0
I20260806 19:42:22.026543 25504 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:42:22.096367 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34833
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:46779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:22.441843 25517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:22.442129 25517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:22.442225 25517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:22.449476 25517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:22.449587 25517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:22.449702 25517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:42:22.459936 25517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:46779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:22.461280 25517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:22.462544 25517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:22.476374 25525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:22.476449 25523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:22.476425 25522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:22.477254 25517 server_base.cc:1034] running on GCE node
I20260806 19:42:22.477981 25517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:22.479157 25517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:22.480383 25517 hybrid_clock.cc:648] HybridClock initialized: now 1786045342480312 us; error 53 us; skew 500 ppm
I20260806 19:42:22.480718 25517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:22 dist-test-slave-77v9 krb5kdc[25408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045342, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:22.484406 25517 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:42:22.486016 25517 webserver.cc:460] Webserver started at http://127.21.202.65:46051/ using document root <none> and password file <none>
I20260806 19:42:22.486726 25517 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:22.486838 25517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:22.487129 25517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:22.490483 25517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "51bf9938ed1f4eaeaf0c3a1750418eb7"
format_stamp: "Formatted at 2026-08-06 19:42:22 on dist-test-slave-77v9"
server_key: "2799df58843c93293eacbc765a166401"
server_key_iv: "b1ade6f18d583c4d56fb9742a57fe564"
server_key_version: "encryptionkey@0"
I20260806 19:42:22.491277 25517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "51bf9938ed1f4eaeaf0c3a1750418eb7"
format_stamp: "Formatted at 2026-08-06 19:42:22 on dist-test-slave-77v9"
server_key: "2799df58843c93293eacbc765a166401"
server_key_iv: "b1ade6f18d583c4d56fb9742a57fe564"
server_key_version: "encryptionkey@0"
I20260806 19:42:22.497370 25517 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 19:42:22.501199 25532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:22.502738 25517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:42:22.502884 25517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "51bf9938ed1f4eaeaf0c3a1750418eb7"
format_stamp: "Formatted at 2026-08-06 19:42:22 on dist-test-slave-77v9"
server_key: "2799df58843c93293eacbc765a166401"
server_key_iv: "b1ade6f18d583c4d56fb9742a57fe564"
server_key_version: "encryptionkey@0"
I20260806 19:42:22.503063 25517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:22.518157 25517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:22.523123 25517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:22.523417 25517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:22.524453 25517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:22.525986 25517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:22.526106 25517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:22.526278 25517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:22.526373 25517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:22.563418 25517 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:41863
I20260806 19:42:22.563459 25647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:41863 every 8 connection(s)
I20260806 19:42:22.564728 25517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:42:22.570861 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 25517
I20260806 19:42:22.571115 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:42:22.571771 22313 external_mini_cluster.cc:1428] Setting key 0db3f572ae16b9031486965c703c4e2b
Aug 06 19:42:22 dist-test-slave-77v9 krb5kdc[25408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045342, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:22.586854 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:22.567283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:41103 (local address 127.21.202.126:34833)
0806 19:42:22.567768 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:22.567772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:22.568589 (+ 817us) server_negotiation.cc:407] Connection header received
0806 19:42:22.570512 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:22.570518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:22.570620 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:22.570744 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:22.572504 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:22.573401 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:22.575259 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:22.576047 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:22.576204 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:22.580136 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:22.580145 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:22.580153 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:22.580198 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:22.583117 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:22.584177 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:22.584182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:22.584185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:22.584323 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:22.584930 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:22.584935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:22.584941 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:22.585246 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:22.585375 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:22.586338 (+ 963us) server_negotiation.cc:299] Negotiation successful
0806 19:42:22.586553 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":275,"threads_started":1}
I20260806 19:42:22.588507 25651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:22.567718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34833 (local address 127.21.202.65:41103)
0806 19:42:22.568422 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:22.568463 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:22.570231 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:22.570982 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:22.571013 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:22.571603 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:22.572224 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:22.572242 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:22.573596 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:22.573603 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:22.575053 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:22.575088 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:22.576299 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:22.576304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:22.576514 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:22.577286 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:22.577310 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:22.579876 (+ 2566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:22.583319 (+ 3443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:22.583332 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:22.583352 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:22.584044 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:22.584455 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:22.584463 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:22.584470 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:22.584729 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:22.585394 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:22.585405 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:22.585900 (+ 495us) client_negotiation.cc:769] Sending connection context
0806 19:42:22.586228 (+ 328us) client_negotiation.cc:240] Negotiation successful
0806 19:42:22.586857 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":274,"threads_started":1}
I20260806 19:42:22.590330 25648 heartbeater.cc:347] Connected to a master server at 127.21.202.126:34833
I20260806 19:42:22.590662 25648 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:22.591488 25648 heartbeater.cc:511] Master 127.21.202.126:34833 requested a full tablet report, sending...
I20260806 19:42:22.594365 25454 ts_manager.cc:194] Registered new tserver with Master: 51bf9938ed1f4eaeaf0c3a1750418eb7 (127.21.202.65:41863)
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
I20260806 19:42:22.596663 25454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:41103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:42:22.628530 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34833
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:46779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:22.962669 25654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:22.962967 25654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:22.963091 25654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:22.970636 25654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:22.970754 25654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:22.970973 25654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:42:22.981618 25654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:46779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:22.983019 25654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:22.984444 25654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:22.997954 25660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:22.997972 25662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:22.998049 25659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:23.002615 25654 server_base.cc:1034] running on GCE node
I20260806 19:42:23.003319 25654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:23.004355 25654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:23.005584 25654 hybrid_clock.cc:648] HybridClock initialized: now 1786045343005514 us; error 54 us; skew 500 ppm
I20260806 19:42:23.005964 25654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:23 dist-test-slave-77v9 krb5kdc[25408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045343, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:23.014037 25654 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:42:23.015946 25654 webserver.cc:460] Webserver started at http://127.21.202.66:33415/ using document root <none> and password file <none>
I20260806 19:42:23.017048 25654 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:23.017226 25654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:23.017575 25654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:23.020697 25654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "d48f5b5bbd484acdb20f266ae845e7d7"
format_stamp: "Formatted at 2026-08-06 19:42:23 on dist-test-slave-77v9"
server_key: "7ad92cb0a4475552d64942b8f5f5dcf5"
server_key_iv: "6fe785658102af9f243c85f898cf59e7"
server_key_version: "encryptionkey@0"
I20260806 19:42:23.021577 25654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "d48f5b5bbd484acdb20f266ae845e7d7"
format_stamp: "Formatted at 2026-08-06 19:42:23 on dist-test-slave-77v9"
server_key: "7ad92cb0a4475552d64942b8f5f5dcf5"
server_key_iv: "6fe785658102af9f243c85f898cf59e7"
server_key_version: "encryptionkey@0"
I20260806 19:42:23.027339 25654 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:42:23.031013 25669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:23.032505 25654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:42:23.032675 25654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "d48f5b5bbd484acdb20f266ae845e7d7"
format_stamp: "Formatted at 2026-08-06 19:42:23 on dist-test-slave-77v9"
server_key: "7ad92cb0a4475552d64942b8f5f5dcf5"
server_key_iv: "6fe785658102af9f243c85f898cf59e7"
server_key_version: "encryptionkey@0"
I20260806 19:42:23.032939 25654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:23.063585 25654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:23.068603 25654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:23.068987 25654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:23.070549 25654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:23.072189 25654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:23.072264 25654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:23.072414 25654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:23.072490 25654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:23.108465 25654 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:38295
I20260806 19:42:23.108556 25786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:38295 every 8 connection(s)
I20260806 19:42:23.109838 25654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:42:23.111835 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 25654
I20260806 19:42:23.112124 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:42:23.112756 22313 external_mini_cluster.cc:1428] Setting key 50f3069a8e6d7f78fc636892dfdff6df
Aug 06 19:42:23 dist-test-slave-77v9 krb5kdc[25408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045343, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:23.133421 25789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:23.113122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:50037 (local address 127.21.202.126:34833)
0806 19:42:23.113645 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:23.113651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:23.114778 (+ 1127us) server_negotiation.cc:407] Connection header received
0806 19:42:23.116670 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:23.116676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:23.116778 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:23.116934 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:23.118444 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.119167 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.120738 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.121230 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.121367 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:23.125660 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:23.125670 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:23.125677 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:23.125724 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:23.129537 (+ 3813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:23.130369 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:23.130378 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:23.130384 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:23.130568 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:23.131059 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:23.131067 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:23.131073 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:23.131482 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:23.131633 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:23.132523 (+ 890us) server_negotiation.cc:299] Negotiation successful
0806 19:42:23.132859 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":270,"threads_started":1}
I20260806 19:42:23.134111 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:23.113726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34833 (local address 127.21.202.66:50037)
0806 19:42:23.114584 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:23.114664 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:23.116403 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:23.117189 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:23.117197 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:23.117707 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:23.118269 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:23.118286 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.119330 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.119334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:23.120591 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:23.120603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.121480 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.121488 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:23.121718 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:23.122504 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:23.122528 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:23.125363 (+ 2835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:23.129718 (+ 4355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:23.129725 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:23.129738 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:23.130213 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:23.130714 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:23.130717 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:23.130719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:23.130930 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:23.131652 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:23.131658 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:23.132116 (+ 458us) client_negotiation.cc:769] Sending connection context
0806 19:42:23.132372 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 19:42:23.132869 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":415,"threads_started":1}
I20260806 19:42:23.135949 25787 heartbeater.cc:347] Connected to a master server at 127.21.202.126:34833
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
I20260806 19:42:23.136343 25787 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:23.137264 25787 heartbeater.cc:511] Master 127.21.202.126:34833 requested a full tablet report, sending...
I20260806 19:42:23.139015 25454 ts_manager.cc:194] Registered new tserver with Master: d48f5b5bbd484acdb20f266ae845e7d7 (127.21.202.66:38295)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
I20260806 19:42:23.139938 25454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:50037
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:42:23.169229 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34833
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:46779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:23.520118 25793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:23.520473 25793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:23.520648 25793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:23.528149 25793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:23.528308 25793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:23.528503 25793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:42:23.538483 25793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:46779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:23.539866 25793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:23.541240 25793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:23.554937 25798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:23.554984 25799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:23.555527 25801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:23.556890 25793 server_base.cc:1034] running on GCE node
I20260806 19:42:23.557664 25793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:23.558758 25793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:23.560021 25793 hybrid_clock.cc:648] HybridClock initialized: now 1786045343559948 us; error 45 us; skew 500 ppm
I20260806 19:42:23.560416 25793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:23 dist-test-slave-77v9 krb5kdc[25408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045343, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:23.563962 25793 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:42:23.565678 25793 webserver.cc:460] Webserver started at http://127.21.202.67:35937/ using document root <none> and password file <none>
I20260806 19:42:23.566445 25793 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:23.566597 25793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:23.567000 25793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:23.570401 25793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "c0a62fe69add4df291e9eb4fffe6c91b"
format_stamp: "Formatted at 2026-08-06 19:42:23 on dist-test-slave-77v9"
server_key: "4278bc813fd20a1cc903db3493ec4d00"
server_key_iv: "5f709821d7e1be5cf781408b9698c766"
server_key_version: "encryptionkey@0"
I20260806 19:42:23.571233 25793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "c0a62fe69add4df291e9eb4fffe6c91b"
format_stamp: "Formatted at 2026-08-06 19:42:23 on dist-test-slave-77v9"
server_key: "4278bc813fd20a1cc903db3493ec4d00"
server_key_iv: "5f709821d7e1be5cf781408b9698c766"
server_key_version: "encryptionkey@0"
I20260806 19:42:23.576920 25793 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 19:42:23.580899 25808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:23.582441 25793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:42:23.582647 25793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "c0a62fe69add4df291e9eb4fffe6c91b"
format_stamp: "Formatted at 2026-08-06 19:42:23 on dist-test-slave-77v9"
server_key: "4278bc813fd20a1cc903db3493ec4d00"
server_key_iv: "5f709821d7e1be5cf781408b9698c766"
server_key_version: "encryptionkey@0"
I20260806 19:42:23.582878 25793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:23.600342 25648 heartbeater.cc:503] Master 127.21.202.126:34833 was elected leader, sending a full tablet report...
I20260806 19:42:23.605695 25793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:23.610558 25793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:23.610890 25793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:23.611980 25793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:23.613586 25793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:23.613696 25793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:23.613867 25793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:23.613950 25793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:23.651014 25793 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:44969
I20260806 19:42:23.651075 25924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:44969 every 8 connection(s)
I20260806 19:42:23.652385 25793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:42:23.659742 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 25793
I20260806 19:42:23.659930 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:42:23.660429 22313 external_mini_cluster.cc:1428] Setting key 685296ab15f82036e329f11eb9c6672a
Aug 06 19:42:23 dist-test-slave-77v9 krb5kdc[25408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045343, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:23.674727 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:23.655124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:39221 (local address 127.21.202.126:34833)
0806 19:42:23.655809 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:23.655814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:23.656611 (+ 797us) server_negotiation.cc:407] Connection header received
0806 19:42:23.658654 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:23.658659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:23.658771 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:23.658913 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:23.660989 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.662004 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.663734 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.664165 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.664304 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:23.668017 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:23.668023 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:23.668026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:23.668064 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:23.670946 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:23.671824 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:23.671829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:23.671831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:23.671968 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:23.672599 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:23.672604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:23.672606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:23.673016 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:23.673157 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:23.674085 (+ 928us) server_negotiation.cc:299] Negotiation successful
0806 19:42:23.674297 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":401,"threads_started":1}
I20260806 19:42:23.676051 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:23.655560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34833 (local address 127.21.202.67:39221)
0806 19:42:23.656417 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:23.656464 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:23.658247 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:23.659169 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:23.659178 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:23.660052 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:23.660743 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:23.660763 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.662175 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.662181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:23.663587 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:23.663599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.664423 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.664431 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:23.664663 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:23.665391 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:23.665420 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:23.667663 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:23.671132 (+ 3469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:23.671142 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:23.671170 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:23.671670 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:23.672088 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:23.672093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:23.672099 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:23.672405 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:23.673192 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:23.673198 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:23.673600 (+ 402us) client_negotiation.cc:769] Sending connection context
0806 19:42:23.673887 (+ 287us) client_negotiation.cc:240] Negotiation successful
0806 19:42:23.674511 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":354,"threads_started":1}
I20260806 19:42:23.677863 25925 heartbeater.cc:347] Connected to a master server at 127.21.202.126:34833
I20260806 19:42:23.678247 25925 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:23.679052 25925 heartbeater.cc:511] Master 127.21.202.126:34833 requested a full tablet report, sending...
I20260806 19:42:23.680668 25454 ts_manager.cc:194] Registered new tserver with Master: c0a62fe69add4df291e9eb4fffe6c91b (127.21.202.67:44969)
I20260806 19:42:23.681649 25454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:39221
I20260806 19:42:23.690729 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 19:42:23 dist-test-slave-77v9 krb5kdc[25408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045343, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 19:42:23 dist-test-slave-77v9 krb5kdc[25408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045343, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:23.751400 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:23.734984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45476 (local address 127.21.202.126:34833)
0806 19:42:23.735270 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:23.735278 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:23.735625 (+ 347us) server_negotiation.cc:407] Connection header received
0806 19:42:23.735924 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:23.735929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:23.736006 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:23.736167 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:23.737947 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.738785 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.740103 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.740602 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.741058 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:23.745369 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:23.745376 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:23.745379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:23.745418 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:23.748200 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:23.749150 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:23.749154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:23.749156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:23.749285 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:23.749734 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:23.749739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:23.749741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:23.750132 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:23.750383 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:23.750751 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 19:42:23.750963 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260806 19:42:23.752089 25454 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45476
I20260806 19:42:23.754311 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 25517
Aug 06 19:42:23 dist-test-slave-77v9 krb5kdc[25408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045343, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:42:23.773051 25938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:23.755588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52106 (local address 127.21.202.65:41863)
0806 19:42:23.756709 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:23.756716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:23.756743 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:42:23.756802 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:23.756865 (+ 63us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:23.757090 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:23.757223 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:23.758665 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.759712 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.761078 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.761657 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.761770 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:23.766195 (+ 4425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:23.766209 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:23.766225 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:23.766272 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:23.769641 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:23.770454 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:23.770464 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:23.770466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:23.770639 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:23.771104 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:23.771111 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:23.771114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:23.771525 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:23.771875 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:23.772235 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 19:42:23.772539 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":773,"threads_started":1}
W20260806 19:42:23.774545 25643 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52106
W20260806 19:42:23.775753 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 25517: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:42:23.775944 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 25517
I20260806 19:42:23.794299 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 25654
Aug 06 19:42:23 dist-test-slave-77v9 krb5kdc[25408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045343, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:42:23.813347 25939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:23.795321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53398 (local address 127.21.202.66:38295)
0806 19:42:23.795978 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:23.795987 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:23.796016 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:42:23.796082 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:23.796089 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:23.796322 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:23.796469 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:23.797917 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.799305 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.800761 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.801499 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.801655 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:23.806428 (+ 4773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:23.806444 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:23.806474 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:23.806518 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:23.809938 (+ 3420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:23.810771 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:23.810779 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:23.810782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:23.810920 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:23.811335 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:23.811341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:23.811344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:23.811723 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:23.812062 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:23.812424 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 19:42:23.812759 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":323,"threads_started":1}
W20260806 19:42:23.814836 25781 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53398
W20260806 19:42:23.815991 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 25654: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:42:23.816155 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 25654
I20260806 19:42:23.832690 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 25793
Aug 06 19:42:23 dist-test-slave-77v9 krb5kdc[25408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045343, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:42:23.851478 25940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:23.833926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52440 (local address 127.21.202.67:44969)
0806 19:42:23.834660 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:23.834667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:23.834697 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:42:23.834759 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:23.834773 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:23.834982 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:23.835122 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:23.836787 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.838108 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.839618 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:23.840228 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:23.840340 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:23.845116 (+ 4776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:23.845130 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:23.845146 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:23.845191 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:23.848412 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:23.849280 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:23.849287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:23.849289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:23.849428 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:23.849866 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:23.849872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:23.849874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:23.850342 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:23.850657 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:23.850957 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 19:42:23.851175 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":406,"threads_started":1}
W20260806 19:42:23.852893 25920 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52440
W20260806 19:42:23.854061 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 25793: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:42:23.854202 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 25793
I20260806 19:42:23.871886 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 25421
I20260806 19:42:23.873323 25485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:23.978232 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 25421
2026-08-06T19:42:23Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3884 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:42:24 dist-test-slave-77v9 krb5kdc[25944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:42:24 dist-test-slave-77v9 krb5kdc[25944](info): set up 2 sockets
Aug 06 19:42:24 dist-test-slave-77v9 krb5kdc[25944](info): commencing operation
krb5kdc: starting...
W20260806 19:42:25.149870 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:42:25 dist-test-slave-77v9 krb5kdc[25944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:42:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:42:25.384712 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:41843
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:41843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:25.731688 25957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:25.732055 25957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:25.732226 25957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:25.739938 25957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:25.740113 25957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:25.740211 25957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:25.740329 25957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:25.740417 25957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:42:25.754547 25957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:41843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:41843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:25.755986 25957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:25.757403 25957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:25.769984 25965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:25.770072 25962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:25.770092 25963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:25.771443 25957 server_base.cc:1034] running on GCE node
I20260806 19:42:25.772367 25957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:25.774034 25957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:25.775298 25957 hybrid_clock.cc:648] HybridClock initialized: now 1786045345775245 us; error 37 us; skew 500 ppm
I20260806 19:42:25.775656 25957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:25 dist-test-slave-77v9 krb5kdc[25944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045345, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:25.779351 25957 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:42:25.781068 25957 webserver.cc:460] Webserver started at http://127.21.202.126:37123/ using document root <none> and password file <none>
I20260806 19:42:25.781836 25957 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:25.781941 25957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:25.782290 25957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:25.785467 25957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "5a4e9847839f4fc3ba61614849f35114"
format_stamp: "Formatted at 2026-08-06 19:42:25 on dist-test-slave-77v9"
server_key: "64afe8d2a828b854012a27fe48ea4f26"
server_key_iv: "bb2cbc110eac83729b78557fd965ed41"
server_key_version: "encryptionkey@0"
I20260806 19:42:25.786314 25957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "5a4e9847839f4fc3ba61614849f35114"
format_stamp: "Formatted at 2026-08-06 19:42:25 on dist-test-slave-77v9"
server_key: "64afe8d2a828b854012a27fe48ea4f26"
server_key_iv: "bb2cbc110eac83729b78557fd965ed41"
server_key_version: "encryptionkey@0"
I20260806 19:42:25.791786 25957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:42:25.795902 25972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:25.797577 25957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:25.797788 25957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "5a4e9847839f4fc3ba61614849f35114"
format_stamp: "Formatted at 2026-08-06 19:42:25 on dist-test-slave-77v9"
server_key: "64afe8d2a828b854012a27fe48ea4f26"
server_key_iv: "bb2cbc110eac83729b78557fd965ed41"
server_key_version: "encryptionkey@0"
I20260806 19:42:25.798035 25957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:25.811370 25957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:25.815996 25957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:25.816322 25957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:25.835595 25957 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:41843
I20260806 19:42:25.835626 26023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:41843 every 8 connection(s)
I20260806 19:42:25.837110 25957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:25.837971 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 25957
I20260806 19:42:25.838219 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:25.838929 22313 external_mini_cluster.cc:1428] Setting key 4e85c2f88202927e2b000dd462c0650c
I20260806 19:42:25.841606 26024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 19:42:25 dist-test-slave-77v9 krb5kdc[25944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:25.852658 26024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114: Bootstrap starting.
I20260806 19:42:25.856228 26024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:25.857813 26024 log.cc:824] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:25.861436 26024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114: No bootstrap required, opened a new log
I20260806 19:42:25.861902 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:25.840960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60532 (local address 127.21.202.126:41843)
0806 19:42:25.841807 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:25.841819 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:25.841860 (+ 41us) server_negotiation.cc:407] Connection header received
0806 19:42:25.843537 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:25.843559 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:25.843927 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:25.844339 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:25.845848 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:25.847255 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:25.848490 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:25.849234 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:25.849465 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:25.853727 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:25.853735 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:25.853750 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:25.853791 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:25.857174 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:25.857924 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:25.857931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:25.857938 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:25.858150 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:25.858650 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:25.858654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:25.858656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:25.859339 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:25.859637 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:25.860063 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 19:42:25.860493 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":360,"threads_started":1}
I20260806 19:42:25.869825 26024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4e9847839f4fc3ba61614849f35114" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41843 } }
I20260806 19:42:25.870178 26024 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:25.870242 26024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a4e9847839f4fc3ba61614849f35114, State: Initialized, Role: FOLLOWER
I20260806 19:42:25.870744 26024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4e9847839f4fc3ba61614849f35114" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41843 } }
I20260806 19:42:25.870954 26024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:25.871042 26024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:25.871166 26024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:25.873719 26024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4e9847839f4fc3ba61614849f35114" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41843 } }
I20260806 19:42:25.874320 26024 leader_election.cc:302] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a4e9847839f4fc3ba61614849f35114; no voters:
I20260806 19:42:25.874979 26024 leader_election.cc:288] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:25.875170 26029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:25.875917 26029 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [term 1 LEADER]: Becoming Leader. State: Replica: 5a4e9847839f4fc3ba61614849f35114, State: Running, Role: LEADER
I20260806 19:42:25.876541 26029 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4e9847839f4fc3ba61614849f35114" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41843 } }
I20260806 19:42:25.877172 26024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:25.880769 26030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a4e9847839f4fc3ba61614849f35114" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4e9847839f4fc3ba61614849f35114" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41843 } } }
I20260806 19:42:25.881201 26030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:25.880522 26031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a4e9847839f4fc3ba61614849f35114. Latest consensus state: current_term: 1 leader_uuid: "5a4e9847839f4fc3ba61614849f35114" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4e9847839f4fc3ba61614849f35114" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41843 } } }
I20260806 19:42:25.881695 26031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:25.883329 26039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:25.888367 26039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:25.897450 26039 catalog_manager.cc:1347] Generated new cluster ID: 5b8acb08076044d39adda8df8b1d5f3e
I20260806 19:42:25.897588 26039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:25.912755 26039 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:25.914746 26039 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:25.929431 26039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a4e9847839f4fc3ba61614849f35114: Generated new TSK 0
I20260806 19:42:25.930619 26039 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:42:25.994688 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41843
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:26.343436 26051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:26.343734 26051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:26.343827 26051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:26.351223 26051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:26.351357 26051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:26.351476 26051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:42:26.361981 26051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:26.363322 26051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:26.364970 26051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:26.378800 26056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:26.378880 26059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:26.378836 26057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:26.379788 26051 server_base.cc:1034] running on GCE node
I20260806 19:42:26.380359 26051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:26.381410 26051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:26.382624 26051 hybrid_clock.cc:648] HybridClock initialized: now 1786045346382598 us; error 41 us; skew 500 ppm
I20260806 19:42:26.382963 26051 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:26 dist-test-slave-77v9 krb5kdc[25944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045346, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:26.386286 26051 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:42:26.387845 26051 webserver.cc:460] Webserver started at http://127.21.202.65:33459/ using document root <none> and password file <none>
I20260806 19:42:26.388559 26051 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:26.388707 26051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:26.389096 26051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:26.392071 26051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "a241c482346a4b58a352cf6c53d447b6"
format_stamp: "Formatted at 2026-08-06 19:42:26 on dist-test-slave-77v9"
server_key: "0f483ac90e93fff264b96ebc706445d5"
server_key_iv: "725321affb0868b5ac399cbcdbb2ae91"
server_key_version: "encryptionkey@0"
I20260806 19:42:26.392943 26051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "a241c482346a4b58a352cf6c53d447b6"
format_stamp: "Formatted at 2026-08-06 19:42:26 on dist-test-slave-77v9"
server_key: "0f483ac90e93fff264b96ebc706445d5"
server_key_iv: "725321affb0868b5ac399cbcdbb2ae91"
server_key_version: "encryptionkey@0"
I20260806 19:42:26.398296 26051 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 19:42:26.401994 26067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:26.403695 26051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:42:26.403883 26051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "a241c482346a4b58a352cf6c53d447b6"
format_stamp: "Formatted at 2026-08-06 19:42:26 on dist-test-slave-77v9"
server_key: "0f483ac90e93fff264b96ebc706445d5"
server_key_iv: "725321affb0868b5ac399cbcdbb2ae91"
server_key_version: "encryptionkey@0"
I20260806 19:42:26.404104 26051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:26.418190 26051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:26.423014 26051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:26.423347 26051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:26.424429 26051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:26.425994 26051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:26.426106 26051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:26.426275 26051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:26.426368 26051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:26.463071 26051 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:41225
I20260806 19:42:26.463135 26182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:41225 every 8 connection(s)
I20260806 19:42:26.464494 26051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:42:26.468159 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 26051
I20260806 19:42:26.468402 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:42:26.469261 22313 external_mini_cluster.cc:1428] Setting key 256210e324b9d5d84e9344965a4e6fff
Aug 06 19:42:26 dist-test-slave-77v9 krb5kdc[25944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045346, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:26.486974 26185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:26.467388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:51439 (local address 127.21.202.126:41843)
0806 19:42:26.467923 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:26.467929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:26.469014 (+ 1085us) server_negotiation.cc:407] Connection header received
0806 19:42:26.471029 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:26.471035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:26.471129 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:26.471255 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:26.472890 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:26.473614 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:26.475283 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:26.475697 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:26.475850 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:26.480359 (+ 4509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:26.480364 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:26.480368 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:26.480404 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:26.483207 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:26.484261 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:26.484266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:26.484268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:26.484418 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:26.484975 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:26.484979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:26.484981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:26.485308 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:26.485442 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:26.486494 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0806 19:42:26.486683 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":242,"threads_started":1}
I20260806 19:42:26.488642 26186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:26.467870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:41843 (local address 127.21.202.65:51439)
0806 19:42:26.468749 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:26.468852 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:26.470723 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:26.471513 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:26.471521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:26.472066 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:26.472663 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:26.472681 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:26.473805 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:26.473809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:26.475105 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:26.475117 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:26.475951 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:26.475959 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:26.476210 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:26.476988 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:26.477012 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:26.480021 (+ 3009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:26.483384 (+ 3363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:26.483397 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:26.483418 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:26.484096 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:26.484538 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:26.484545 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:26.484552 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:26.484845 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:26.485454 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:26.485465 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:26.485964 (+ 499us) client_negotiation.cc:769] Sending connection context
0806 19:42:26.486312 (+ 348us) client_negotiation.cc:240] Negotiation successful
0806 19:42:26.486994 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":352,"threads_started":1}
I20260806 19:42:26.490698 26183 heartbeater.cc:347] Connected to a master server at 127.21.202.126:41843
I20260806 19:42:26.491091 26183 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:26.492221 26183 heartbeater.cc:511] Master 127.21.202.126:41843 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
I20260806 19:42:26.495055 25989 ts_manager.cc:194] Registered new tserver with Master: a241c482346a4b58a352cf6c53d447b6 (127.21.202.65:41225)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
I20260806 19:42:26.496935 25989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:51439
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:42:26.525841 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41843
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:26.889914 26189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:26.890189 26189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:26.890296 26189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:26.897388 26189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:26.897513 26189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:26.897625 26189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:42:26.910887 26189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:26.912693 26189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:26.914665 26189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:26.929418 26198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:26.929446 26196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:26.929354 26195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:26.930481 26189 server_base.cc:1034] running on GCE node
I20260806 19:42:26.931156 26189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:26.932186 26189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:26.933441 26189 hybrid_clock.cc:648] HybridClock initialized: now 1786045346933375 us; error 62 us; skew 500 ppm
I20260806 19:42:26.933792 26189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:26 dist-test-slave-77v9 krb5kdc[25944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045346, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:26.937131 26189 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:42:26.938658 26189 webserver.cc:460] Webserver started at http://127.21.202.66:41351/ using document root <none> and password file <none>
I20260806 19:42:26.939420 26189 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:26.939525 26189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:26.939790 26189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:26.942732 26189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "df17b4a89dca4f4b9150976c6c1e2c83"
format_stamp: "Formatted at 2026-08-06 19:42:26 on dist-test-slave-77v9"
server_key: "af678b45d4e83bcd5749ae865d110f48"
server_key_iv: "ea6aa9352a6dbeb45c1d223a2b2454c6"
server_key_version: "encryptionkey@0"
I20260806 19:42:26.943490 26189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "df17b4a89dca4f4b9150976c6c1e2c83"
format_stamp: "Formatted at 2026-08-06 19:42:26 on dist-test-slave-77v9"
server_key: "af678b45d4e83bcd5749ae865d110f48"
server_key_iv: "ea6aa9352a6dbeb45c1d223a2b2454c6"
server_key_version: "encryptionkey@0"
I20260806 19:42:26.948803 26189 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 19:42:26.952461 26205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:26.953974 26189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:26.954115 26189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "df17b4a89dca4f4b9150976c6c1e2c83"
format_stamp: "Formatted at 2026-08-06 19:42:26 on dist-test-slave-77v9"
server_key: "af678b45d4e83bcd5749ae865d110f48"
server_key_iv: "ea6aa9352a6dbeb45c1d223a2b2454c6"
server_key_version: "encryptionkey@0"
I20260806 19:42:26.954301 26189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:26.971760 26189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:26.975934 26189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:26.976179 26189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:26.977229 26189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:26.978667 26189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:26.978755 26189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:26.978894 26189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:26.978942 26189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:27.014416 26189 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:35273
I20260806 19:42:27.014467 26319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:35273 every 8 connection(s)
I20260806 19:42:27.015724 26189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:42:27.020968 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 26189
I20260806 19:42:27.021164 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:42:27.021654 22313 external_mini_cluster.cc:1428] Setting key 854da16ffec211e77d6384ac773b2562
Aug 06 19:42:27 dist-test-slave-77v9 krb5kdc[25944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045346, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:27.036458 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:27.018478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:48337 (local address 127.21.202.126:41843)
0806 19:42:27.019013 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:27.019018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:27.020000 (+ 982us) server_negotiation.cc:407] Connection header received
0806 19:42:27.022249 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:27.022255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:27.022356 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:27.022481 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:27.024245 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.025031 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.026560 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.027005 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.027141 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:27.030652 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:27.030657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:27.030660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:27.030691 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:27.033417 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.034135 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.034142 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.034148 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.034277 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.034709 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.034715 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.034718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.034977 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:27.035138 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:27.035822 (+ 684us) server_negotiation.cc:299] Negotiation successful
0806 19:42:27.036057 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":278,"threads_started":1}
I20260806 19:42:27.037518 26323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:27.019051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:41843 (local address 127.21.202.66:48337)
0806 19:42:27.019832 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:27.019872 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:27.021906 (+ 2034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:27.022751 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:27.022762 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:27.023476 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:27.024072 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:27.024089 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.025180 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.025186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:27.026401 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:27.026412 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.027238 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.027246 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:27.027505 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:27.028202 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:27.028230 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:27.030418 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:27.033562 (+ 3144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:27.033571 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:27.033588 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:27.034011 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:27.034397 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:27.034402 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:27.034405 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:27.034592 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:27.035117 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:27.035125 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:27.035465 (+ 340us) client_negotiation.cc:769] Sending connection context
0806 19:42:27.035743 (+ 278us) client_negotiation.cc:240] Negotiation successful
0806 19:42:27.036289 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":326,"threads_started":1}
I20260806 19:42:27.039068 26320 heartbeater.cc:347] Connected to a master server at 127.21.202.126:41843
I20260806 19:42:27.039388 26320 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:27.040158 26320 heartbeater.cc:511] Master 127.21.202.126:41843 requested a full tablet report, sending...
I20260806 19:42:27.041716 25989 ts_manager.cc:194] Registered new tserver with Master: df17b4a89dca4f4b9150976c6c1e2c83 (127.21.202.66:35273)
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
I20260806 19:42:27.042584 25989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:48337
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:42:27.073956 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41843
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:27.420114 26326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:27.420415 26326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:27.420509 26326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:27.429596 26326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:27.429771 26326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:27.430065 26326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:42:27.441987 26326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:27.443321 26326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:27.444578 26326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:27.458451 26331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:27.458534 26332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:27.458715 26335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:27.459609 26326 server_base.cc:1034] running on GCE node
I20260806 19:42:27.460292 26326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:27.461364 26326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:27.462630 26326 hybrid_clock.cc:648] HybridClock initialized: now 1786045347462582 us; error 62 us; skew 500 ppm
I20260806 19:42:27.463040 26326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:27 dist-test-slave-77v9 krb5kdc[25944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045347, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:27.467304 26326 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:42:27.470203 26326 webserver.cc:460] Webserver started at http://127.21.202.67:34529/ using document root <none> and password file <none>
I20260806 19:42:27.470953 26326 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:27.471107 26326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:27.471424 26326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:27.474563 26326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "0aacc13757be4ab897e9de4222dc4197"
format_stamp: "Formatted at 2026-08-06 19:42:27 on dist-test-slave-77v9"
server_key: "b1dfda5e885b1877d6feec97e20996da"
server_key_iv: "f8e2e72d745f2592e937feee1e2afa34"
server_key_version: "encryptionkey@0"
I20260806 19:42:27.475407 26326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "0aacc13757be4ab897e9de4222dc4197"
format_stamp: "Formatted at 2026-08-06 19:42:27 on dist-test-slave-77v9"
server_key: "b1dfda5e885b1877d6feec97e20996da"
server_key_iv: "f8e2e72d745f2592e937feee1e2afa34"
server_key_version: "encryptionkey@0"
I20260806 19:42:27.481288 26326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 19:42:27.485136 26342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:27.486673 26326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:42:27.486863 26326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "0aacc13757be4ab897e9de4222dc4197"
format_stamp: "Formatted at 2026-08-06 19:42:27 on dist-test-slave-77v9"
server_key: "b1dfda5e885b1877d6feec97e20996da"
server_key_iv: "f8e2e72d745f2592e937feee1e2afa34"
server_key_version: "encryptionkey@0"
I20260806 19:42:27.487098 26326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:27.500128 26183 heartbeater.cc:503] Master 127.21.202.126:41843 was elected leader, sending a full tablet report...
I20260806 19:42:27.505067 26326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:27.509820 26326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:27.510149 26326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:27.511243 26326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:27.512987 26326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:27.513196 26326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:27.513358 26326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:27.513442 26326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:27.547865 26326 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:38265
I20260806 19:42:27.547977 26458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:38265 every 8 connection(s)
I20260806 19:42:27.549240 26326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:42:27.557693 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 26326
I20260806 19:42:27.557894 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:42:27.558497 22313 external_mini_cluster.cc:1428] Setting key 9bf5f074a271325dfcd4c6bdc823bcf0
Aug 06 19:42:27 dist-test-slave-77v9 krb5kdc[25944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045347, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:27.571722 26461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:27.552149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:42671 (local address 127.21.202.126:41843)
0806 19:42:27.552687 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:27.552694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:27.553727 (+ 1033us) server_negotiation.cc:407] Connection header received
0806 19:42:27.555539 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:27.555544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:27.555629 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:27.555732 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:27.557333 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.558611 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.560712 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.561407 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.561515 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:27.565385 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:27.565390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:27.565394 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:27.565432 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:27.568382 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.569247 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.569260 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.569262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.569401 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.569791 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.569795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.569797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.570289 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:27.570437 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:27.571229 (+ 792us) server_negotiation.cc:299] Negotiation successful
0806 19:42:27.571417 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":247,"threads_started":1}
I20260806 19:42:27.572718 26462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:27.552697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:41843 (local address 127.21.202.67:42671)
0806 19:42:27.553534 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:27.553574 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:27.555256 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:27.555946 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:27.555954 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:27.556480 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:27.557135 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:27.557153 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.558786 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.558790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:27.560513 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:27.560529 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.561660 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.561668 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:27.562043 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:27.562797 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:27.562826 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:27.565144 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:27.568529 (+ 3385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:27.568536 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:27.568549 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:27.569095 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:27.569505 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:27.569508 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:27.569511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:27.569687 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:27.570480 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:27.570487 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:27.570823 (+ 336us) client_negotiation.cc:769] Sending connection context
0806 19:42:27.571077 (+ 254us) client_negotiation.cc:240] Negotiation successful
0806 19:42:27.571480 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":297,"threads_started":1}
I20260806 19:42:27.574935 26459 heartbeater.cc:347] Connected to a master server at 127.21.202.126:41843
I20260806 19:42:27.575266 26459 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:27.576108 26459 heartbeater.cc:511] Master 127.21.202.126:41843 requested a full tablet report, sending...
I20260806 19:42:27.577637 25989 ts_manager.cc:194] Registered new tserver with Master: 0aacc13757be4ab897e9de4222dc4197 (127.21.202.67:38265)
I20260806 19:42:27.578562 25989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:42671
I20260806 19:42:27.587561 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 19:42:27 dist-test-slave-77v9 krb5kdc[25944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 19:42:27 dist-test-slave-77v9 krb5kdc[25944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:42:27.643570 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:27.627054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56710 (local address 127.21.202.65:41225)
0806 19:42:27.627793 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:27.627801 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:27.627837 (+ 36us) server_negotiation.cc:407] Connection header received
0806 19:42:27.627902 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:27.627908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:27.628124 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:27.628255 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:27.629793 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.630899 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.632181 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.632865 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.632977 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:27.637112 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:27.637124 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:27.637139 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:27.637180 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:27.640369 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.641247 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.641255 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.641258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.641417 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.641838 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.641842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.641845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.642260 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:27.642532 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:27.642872 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 19:42:27.643099 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":439,"threads_started":1}
Aug 06 19:42:27 dist-test-slave-77v9 krb5kdc[25944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:27.663980 26461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:27.648706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60544 (local address 127.21.202.126:41843)
0806 19:42:27.649004 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:27.649011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:27.649209 (+ 198us) server_negotiation.cc:407] Connection header received
0806 19:42:27.649453 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:27.649459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:27.649550 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:27.649683 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:27.651176 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.652006 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.653475 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.653924 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.654079 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:27.658500 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:27.658507 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:27.658512 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:27.658559 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:27.661205 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.661879 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.661883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.661885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.662004 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.662414 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.662418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.662420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.662745 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:27.663024 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:27.663375 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 19:42:27.663609 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
W20260806 19:42:27.664605 25989 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60544
I20260806 19:42:27.665887 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 26051
I20260806 19:42:27.679795 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:27.667019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.21.202.65:41225)
0806 19:42:27.667256 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:27.667264 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:27.667360 (+ 96us) server_negotiation.cc:407] Connection header received
0806 19:42:27.667497 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:27.667501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:27.667571 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:27.667658 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:27.668937 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.669688 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.670973 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.671485 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.671601 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:27.673958 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:27.673964 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:27.673967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:27.674013 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:27.676790 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.677551 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.677561 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.677566 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.677777 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.678232 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.678237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.678239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.678529 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:27.678846 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:27.679218 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 19:42:27.679392 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 19:42:27.680544 26178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:27.766574 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 26051
I20260806 19:42:27.784651 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 26189
Aug 06 19:42:27 dist-test-slave-77v9 krb5kdc[25944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:42:27.802774 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:27.786137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54600 (local address 127.21.202.66:35273)
0806 19:42:27.786763 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:27.786770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:27.786797 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:42:27.786903 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:27.786909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:27.787124 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:27.787321 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:27.788674 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.789944 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.791390 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.791986 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.792133 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:27.796476 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:27.796489 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:27.796504 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:27.796541 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:27.799711 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.800459 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.800468 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.800470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.800597 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.801167 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.801170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.801172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.801517 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:27.801817 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:27.802142 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 19:42:27.802340 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":331,"threads_started":1}
I20260806 19:42:27.804064 26315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:27.888218 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 26189
I20260806 19:42:27.905949 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 26326
Aug 06 19:42:27 dist-test-slave-77v9 krb5kdc[25944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:42:27.923696 26474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:27.906964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48418 (local address 127.21.202.67:38265)
0806 19:42:27.907688 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:27.907696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:27.907728 (+ 32us) server_negotiation.cc:407] Connection header received
0806 19:42:27.907795 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:27.907802 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:27.908037 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:27.908178 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:27.909546 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.910744 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.912255 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:27.912941 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:27.913069 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:27.917848 (+ 4779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:27.917862 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:27.917877 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:27.917920 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:27.920770 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.921620 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.921630 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.921632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.921759 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:27.922197 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:27.922201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:27.922203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:27.922566 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:27.922837 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:27.923169 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 19:42:27.923392 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":365,"threads_started":1}
I20260806 19:42:27.925364 26454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:28.009423 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 26326
I20260806 19:42:28.026979 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 25957
I20260806 19:42:28.028314 26019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:28.134508 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 25957
2026-08-06T19:42:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4157 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:42:28 dist-test-slave-77v9 krb5kdc[26479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:42:28 dist-test-slave-77v9 krb5kdc[26479](info): set up 2 sockets
Aug 06 19:42:28 dist-test-slave-77v9 krb5kdc[26479](info): commencing operation
krb5kdc: starting...
W20260806 19:42:29.281598 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.007s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:42:29 dist-test-slave-77v9 krb5kdc[26479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:42:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:42:29.530138 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:37487
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:37487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:29.875336 26492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:29.875708 26492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:29.875880 26492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:29.883404 26492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:29.883558 26492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:29.883637 26492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:29.883728 26492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:29.883797 26492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:42:29.894579 26492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:37487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:37487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:29.896044 26492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:29.897437 26492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:29.909476 26497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:29.909374 26498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:29.909325 26500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:29.910377 26492 server_base.cc:1034] running on GCE node
I20260806 19:42:29.911221 26492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:29.912781 26492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:29.914089 26492 hybrid_clock.cc:648] HybridClock initialized: now 1786045349914047 us; error 37 us; skew 500 ppm
I20260806 19:42:29.914507 26492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:29 dist-test-slave-77v9 krb5kdc[26479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045349, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:29.918009 26492 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:42:29.919317 26492 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:29.919472 26492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:29.919826 26492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:29.923076 26492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "10d2519f5b7045588fa255d92facdb06"
format_stamp: "Formatted at 2026-08-06 19:42:29 on dist-test-slave-77v9"
server_key: "11cb83221eefd6b62928bcb5c3a09187"
server_key_iv: "99c519342f674788f1bd48db2aee4831"
server_key_version: "encryptionkey@0"
I20260806 19:42:29.923933 26492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "10d2519f5b7045588fa255d92facdb06"
format_stamp: "Formatted at 2026-08-06 19:42:29 on dist-test-slave-77v9"
server_key: "11cb83221eefd6b62928bcb5c3a09187"
server_key_iv: "99c519342f674788f1bd48db2aee4831"
server_key_version: "encryptionkey@0"
I20260806 19:42:29.930608 26492 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 19:42:29.935098 26506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:29.937144 26492 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:42:29.937362 26492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "10d2519f5b7045588fa255d92facdb06"
format_stamp: "Formatted at 2026-08-06 19:42:29 on dist-test-slave-77v9"
server_key: "11cb83221eefd6b62928bcb5c3a09187"
server_key_iv: "99c519342f674788f1bd48db2aee4831"
server_key_version: "encryptionkey@0"
I20260806 19:42:29.937613 26492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:29.998909 26492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:30.004379 26492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:30.004703 26492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:30.026623 26492 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:37487
I20260806 19:42:30.026644 26559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:37487 every 8 connection(s)
I20260806 19:42:30.027629 26492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:30.031615 26560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:30.035256 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 26492
I20260806 19:42:30.035473 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:30.036316 22313 external_mini_cluster.cc:1428] Setting key 3be1a90834c5fc9c0302969fe98abbad
I20260806 19:42:30.045863 26560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06: Bootstrap starting.
I20260806 19:42:30.050100 26560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:30.051502 26560 log.cc:824] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06: Log is configured to *not* fsync() on all Append() calls
Aug 06 19:42:30 dist-test-slave-77v9 krb5kdc[26479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:30.054646 26560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06: No bootstrap required, opened a new log
I20260806 19:42:30.061267 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:30.038691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43142 (local address 127.21.202.126:37487)
0806 19:42:30.039563 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:30.039575 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:30.039619 (+ 44us) server_negotiation.cc:407] Connection header received
0806 19:42:30.041374 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:30.041398 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:30.041947 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:30.042315 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:30.044246 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:30.045707 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:30.047506 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:30.048226 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:30.048384 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:30.053572 (+ 5188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:30.053580 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:30.053597 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:30.053638 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:30.056759 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:30.057552 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:30.057568 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:30.057575 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:30.057768 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:30.058196 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:30.058201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:30.058206 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:30.058779 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:30.059129 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:30.059617 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 19:42:30.060082 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":438,"threads_started":1}
I20260806 19:42:30.062750 26560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10d2519f5b7045588fa255d92facdb06" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37487 } }
I20260806 19:42:30.063130 26560 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:30.063261 26560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10d2519f5b7045588fa255d92facdb06, State: Initialized, Role: FOLLOWER
I20260806 19:42:30.063865 26560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10d2519f5b7045588fa255d92facdb06" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37487 } }
I20260806 19:42:30.064129 26560 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:30.064278 26560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:30.064487 26560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:30.066802 26560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10d2519f5b7045588fa255d92facdb06" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37487 } }
I20260806 19:42:30.067260 26560 leader_election.cc:302] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10d2519f5b7045588fa255d92facdb06; no voters:
I20260806 19:42:30.067868 26560 leader_election.cc:288] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:30.068032 26565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:30.069379 26565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [term 1 LEADER]: Becoming Leader. State: Replica: 10d2519f5b7045588fa255d92facdb06, State: Running, Role: LEADER
I20260806 19:42:30.069900 26565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10d2519f5b7045588fa255d92facdb06" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37487 } }
I20260806 19:42:30.070307 26560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:30.072799 26567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10d2519f5b7045588fa255d92facdb06. Latest consensus state: current_term: 1 leader_uuid: "10d2519f5b7045588fa255d92facdb06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10d2519f5b7045588fa255d92facdb06" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37487 } } }
I20260806 19:42:30.073302 26567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:30.073868 26566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10d2519f5b7045588fa255d92facdb06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10d2519f5b7045588fa255d92facdb06" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37487 } } }
I20260806 19:42:30.074218 26566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:30.076644 26575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:30.081681 26575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:30.090654 26575 catalog_manager.cc:1347] Generated new cluster ID: e406311075ac4379a76626ef62d9962d
I20260806 19:42:30.090783 26575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:30.104147 26575 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:30.105706 26575 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:30.112151 26575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06: Generated new TSK 0
I20260806 19:42:30.113272 26575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:42:30.177513 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:37487
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:30.525641 26587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:30.525951 26587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:30.526054 26587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:30.533595 26587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:30.533712 26587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:30.533829 26587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:42:30.544026 26587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:37487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:30.545442 26587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:30.546770 26587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:30.559715 26595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:30.559746 26592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:30.559780 26593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:30.562053 26587 server_base.cc:1034] running on GCE node
I20260806 19:42:30.562651 26587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:30.563616 26587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:30.564877 26587 hybrid_clock.cc:648] HybridClock initialized: now 1786045350564760 us; error 56 us; skew 500 ppm
I20260806 19:42:30.565227 26587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:30 dist-test-slave-77v9 krb5kdc[26479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045350, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:30.568440 26587 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:42:30.569748 26587 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:30.569890 26587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:30.570492 26587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:30.574201 26587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "9c038924954a48a6a2aa054b795f1264"
format_stamp: "Formatted at 2026-08-06 19:42:30 on dist-test-slave-77v9"
server_key: "3ed30b044a3e3d5a1d5ac0e0cc6c84d2"
server_key_iv: "1fea286bf583547a3a168104874f22e6"
server_key_version: "encryptionkey@0"
I20260806 19:42:30.575327 26587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "9c038924954a48a6a2aa054b795f1264"
format_stamp: "Formatted at 2026-08-06 19:42:30 on dist-test-slave-77v9"
server_key: "3ed30b044a3e3d5a1d5ac0e0cc6c84d2"
server_key_iv: "1fea286bf583547a3a168104874f22e6"
server_key_version: "encryptionkey@0"
I20260806 19:42:30.580858 26587 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260806 19:42:30.584489 26603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:30.586024 26587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:30.586213 26587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "9c038924954a48a6a2aa054b795f1264"
format_stamp: "Formatted at 2026-08-06 19:42:30 on dist-test-slave-77v9"
server_key: "3ed30b044a3e3d5a1d5ac0e0cc6c84d2"
server_key_iv: "1fea286bf583547a3a168104874f22e6"
server_key_version: "encryptionkey@0"
I20260806 19:42:30.586481 26587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:30.613467 26587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:30.618000 26587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:30.618304 26587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:30.619302 26587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:30.620700 26587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:30.620880 26587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:30.621021 26587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:30.621110 26587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:30.658921 26587 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:42961
I20260806 19:42:30.658973 26717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:42961 every 8 connection(s)
I20260806 19:42:30.659855 26587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:42:30.664110 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 26587
I20260806 19:42:30.664371 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:42:30.665297 22313 external_mini_cluster.cc:1428] Setting key 14f9212e601417703770eacae646aef8
Aug 06 19:42:30 dist-test-slave-77v9 krb5kdc[26479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045350, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:30.682331 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:30.662742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:52347 (local address 127.21.202.126:37487)
0806 19:42:30.663261 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:30.663267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:30.664377 (+ 1110us) server_negotiation.cc:407] Connection header received
0806 19:42:30.666621 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:30.666626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:30.666715 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:30.666823 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:30.668598 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:30.669400 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:30.671197 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:30.671592 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:30.671801 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:30.675944 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:30.675950 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:30.675954 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:30.675990 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:30.678922 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:30.679726 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:30.679744 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:30.679747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:30.679915 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:30.680387 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:30.680394 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:30.680397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:30.680706 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:30.680914 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:30.681656 (+ 742us) server_negotiation.cc:299] Negotiation successful
0806 19:42:30.681916 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":268,"threads_started":1}
I20260806 19:42:30.683437 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:30.663319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:37487 (local address 127.21.202.65:52347)
0806 19:42:30.664180 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:30.664254 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:30.666289 (+ 2035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:30.667092 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:30.667104 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:30.667832 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:30.668423 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:30.668441 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:30.669577 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:30.669582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:30.671036 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:30.671049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:30.671890 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:30.671900 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:30.672141 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:30.672999 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:30.673024 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:30.675675 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:30.679081 (+ 3406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:30.679092 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:30.679110 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:30.679584 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:30.680044 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:30.680051 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:30.680055 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:30.680263 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:30.680872 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:30.680882 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:30.681261 (+ 379us) client_negotiation.cc:769] Sending connection context
0806 19:42:30.681556 (+ 295us) client_negotiation.cc:240] Negotiation successful
0806 19:42:30.682153 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":405,"threads_started":1}
I20260806 19:42:30.685245 26718 heartbeater.cc:347] Connected to a master server at 127.21.202.126:37487
I20260806 19:42:30.685596 26718 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:30.686437 26718 heartbeater.cc:511] Master 127.21.202.126:37487 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
I20260806 19:42:30.688881 26525 ts_manager.cc:194] Registered new tserver with Master: 9c038924954a48a6a2aa054b795f1264 (127.21.202.65:42961)
I20260806 19:42:30.690299 26525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:52347
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:42:30.720309 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:37487
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:31.060379 26724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:31.060667 26724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:31.060765 26724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:31.068346 26724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:31.068470 26724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:31.068589 26724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:42:31.079144 26724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:37487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:31.080470 26724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:31.081833 26724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:31.095402 26729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:31.095428 26730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:31.095355 26732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:31.096277 26724 server_base.cc:1034] running on GCE node
I20260806 19:42:31.097019 26724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:31.097994 26724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:31.099237 26724 hybrid_clock.cc:648] HybridClock initialized: now 1786045351099184 us; error 45 us; skew 500 ppm
I20260806 19:42:31.099588 26724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:31 dist-test-slave-77v9 krb5kdc[26479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:31.102916 26724 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:42:31.104092 26724 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:31.104230 26724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:31.104553 26724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:31.107808 26724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "893a11884d024e26a70492d59639aced"
format_stamp: "Formatted at 2026-08-06 19:42:31 on dist-test-slave-77v9"
server_key: "a42b76af4ee4988221fd80ad5d482337"
server_key_iv: "a89920bfb9922956d36494d823db1df5"
server_key_version: "encryptionkey@0"
I20260806 19:42:31.108661 26724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "893a11884d024e26a70492d59639aced"
format_stamp: "Formatted at 2026-08-06 19:42:31 on dist-test-slave-77v9"
server_key: "a42b76af4ee4988221fd80ad5d482337"
server_key_iv: "a89920bfb9922956d36494d823db1df5"
server_key_version: "encryptionkey@0"
I20260806 19:42:31.114140 26724 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 19:42:31.117762 26738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:31.119144 26724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:42:31.119334 26724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "893a11884d024e26a70492d59639aced"
format_stamp: "Formatted at 2026-08-06 19:42:31 on dist-test-slave-77v9"
server_key: "a42b76af4ee4988221fd80ad5d482337"
server_key_iv: "a89920bfb9922956d36494d823db1df5"
server_key_version: "encryptionkey@0"
I20260806 19:42:31.119580 26724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:31.134620 26724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:31.139042 26724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:31.139338 26724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:31.140326 26724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:31.141861 26724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:31.141981 26724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:31.142115 26724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:31.142200 26724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:31.177757 26724 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:35723
I20260806 19:42:31.177811 26852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:35723 every 8 connection(s)
I20260806 19:42:31.178733 26724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:42:31.184214 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 26724
I20260806 19:42:31.184437 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:42:31.184981 22313 external_mini_cluster.cc:1428] Setting key 8e015c8564ceb2a80bd7aa877762091d
Aug 06 19:42:31 dist-test-slave-77v9 krb5kdc[26479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:31.201478 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.181337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:45397 (local address 127.21.202.126:37487)
0806 19:42:31.181528 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:31.181532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:31.182981 (+ 1449us) server_negotiation.cc:407] Connection header received
0806 19:42:31.185492 (+ 2511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:31.185498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:31.185612 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:31.185741 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:31.187560 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.188263 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.190031 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.190545 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.190687 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.194795 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:31.194801 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:31.194804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:31.194839 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:31.197985 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:31.198938 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:31.198946 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:31.198949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:31.199095 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:31.199589 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:31.199593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:31.199595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:31.199884 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:31.200063 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:31.200875 (+ 812us) server_negotiation.cc:299] Negotiation successful
0806 19:42:31.201122 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 19:42:31.202574 26855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.181891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:37487 (local address 127.21.202.66:45397)
0806 19:42:31.182783 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:31.182831 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:31.185152 (+ 2321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:31.186136 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:31.186147 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:31.186721 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:31.187353 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.187373 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.188448 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.188452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.189864 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.189878 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.190787 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.190795 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.191043 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.191767 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:31.191795 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:31.194509 (+ 2714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:31.198181 (+ 3672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:31.198192 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:31.198210 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:31.198706 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:31.199219 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:31.199224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:31.199228 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:31.199444 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:31.200021 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:31.200030 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:31.200419 (+ 389us) client_negotiation.cc:769] Sending connection context
0806 19:42:31.200699 (+ 280us) client_negotiation.cc:240] Negotiation successful
0806 19:42:31.201324 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":366,"threads_started":1}
I20260806 19:42:31.204317 26853 heartbeater.cc:347] Connected to a master server at 127.21.202.126:37487
I20260806 19:42:31.204660 26853 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:31.205410 26853 heartbeater.cc:511] Master 127.21.202.126:37487 requested a full tablet report, sending...
I20260806 19:42:31.207067 26525 ts_manager.cc:194] Registered new tserver with Master: 893a11884d024e26a70492d59639aced (127.21.202.66:35723)
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
I20260806 19:42:31.208036 26525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:45397
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:42:31.238452 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:37487
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:31.589571 26858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:31.589902 26858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:31.590041 26858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:31.597581 26858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:31.597702 26858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:31.597833 26858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:42:31.607758 26858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:37487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:31.609472 26858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:31.610888 26858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:31.624845 26864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:31.625074 26863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:31.625108 26866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:31.626458 26858 server_base.cc:1034] running on GCE node
I20260806 19:42:31.627178 26858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:31.628259 26858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:31.629520 26858 hybrid_clock.cc:648] HybridClock initialized: now 1786045351629479 us; error 57 us; skew 500 ppm
I20260806 19:42:31.629931 26858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:31 dist-test-slave-77v9 krb5kdc[26479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:31.633276 26858 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:42:31.634560 26858 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:31.634663 26858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:31.634941 26858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:31.638024 26858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "a0d65c79c80b4119b6866390ac9fc099"
format_stamp: "Formatted at 2026-08-06 19:42:31 on dist-test-slave-77v9"
server_key: "98ed5c8008873002273be7945a94cf92"
server_key_iv: "639a4d6dc4b56de9868beecdc5e9a7e6"
server_key_version: "encryptionkey@0"
I20260806 19:42:31.638818 26858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "a0d65c79c80b4119b6866390ac9fc099"
format_stamp: "Formatted at 2026-08-06 19:42:31 on dist-test-slave-77v9"
server_key: "98ed5c8008873002273be7945a94cf92"
server_key_iv: "639a4d6dc4b56de9868beecdc5e9a7e6"
server_key_version: "encryptionkey@0"
I20260806 19:42:31.644649 26858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 19:42:31.648573 26873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:31.650234 26858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:31.650398 26858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "a0d65c79c80b4119b6866390ac9fc099"
format_stamp: "Formatted at 2026-08-06 19:42:31 on dist-test-slave-77v9"
server_key: "98ed5c8008873002273be7945a94cf92"
server_key_iv: "639a4d6dc4b56de9868beecdc5e9a7e6"
server_key_version: "encryptionkey@0"
I20260806 19:42:31.650615 26858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:31.662747 26858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:31.667204 26858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:31.667515 26858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:31.668557 26858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:31.670045 26858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:31.670135 26858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:31.670315 26858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:31.670403 26858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:31.693591 26718 heartbeater.cc:503] Master 127.21.202.126:37487 was elected leader, sending a full tablet report...
I20260806 19:42:31.708226 26858 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:34803
I20260806 19:42:31.708353 26987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:34803 every 8 connection(s)
I20260806 19:42:31.709175 26858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:42:31.718963 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 26858
I20260806 19:42:31.719172 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:42:31.719643 22313 external_mini_cluster.cc:1428] Setting key b2c776aa22ad1a280d11cdbe70bee5b8
Aug 06 19:42:31 dist-test-slave-77v9 krb5kdc[26479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:31.731693 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.712016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:47253 (local address 127.21.202.126:37487)
0806 19:42:31.712491 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:31.712495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:31.713563 (+ 1068us) server_negotiation.cc:407] Connection header received
0806 19:42:31.715940 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:31.715946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:31.716040 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:31.716176 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:31.717899 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.718860 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.720743 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.721672 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.721838 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.725830 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:31.725836 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:31.725839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:31.725876 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:31.728666 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:31.729440 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:31.729444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:31.729446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:31.729564 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:31.730053 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:31.730057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:31.730059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:31.730330 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:31.730450 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:31.731212 (+ 762us) server_negotiation.cc:299] Negotiation successful
0806 19:42:31.731398 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":256,"threads_started":1}
I20260806 19:42:31.732730 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.712511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:37487 (local address 127.21.202.67:47253)
0806 19:42:31.713401 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:31.713449 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:31.715658 (+ 2209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:31.716463 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:31.716477 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:31.717130 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:31.717705 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.717721 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.719076 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.719083 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.720538 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.720552 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.721983 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.721992 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.722315 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.723094 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:31.723128 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:31.725594 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:31.728801 (+ 3207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:31.728854 (+ 53us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:31.728874 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:31.729319 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:31.729685 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:31.729689 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:31.729691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:31.729914 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:31.730460 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:31.730466 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:31.730841 (+ 375us) client_negotiation.cc:769] Sending connection context
0806 19:42:31.731081 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 19:42:31.731521 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":365,"threads_started":1}
I20260806 19:42:31.734431 26988 heartbeater.cc:347] Connected to a master server at 127.21.202.126:37487
I20260806 19:42:31.734787 26988 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:31.735563 26988 heartbeater.cc:511] Master 127.21.202.126:37487 requested a full tablet report, sending...
I20260806 19:42:31.737249 26525 ts_manager.cc:194] Registered new tserver with Master: a0d65c79c80b4119b6866390ac9fc099 (127.21.202.67:34803)
I20260806 19:42:31.738039 26525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:47253
I20260806 19:42:31.748239 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:42:31.766036 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.753151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43146 (local address 127.21.202.126:37487)
0806 19:42:31.753348 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:31.753352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:31.753872 (+ 520us) server_negotiation.cc:407] Connection header received
0806 19:42:31.754078 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:31.754082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:31.754151 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:31.754243 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:31.755834 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.756577 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.758068 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.758491 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.758596 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.760921 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:31.760926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:31.760930 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:31.760963 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:31.763656 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:31.764264 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:31.764269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:31.764271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:31.764404 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:31.764721 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:31.764724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:31.764726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:31.765110 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:31.765291 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:31.765634 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 19:42:31.765798 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 19:42:31.770421 26525 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:42:31.778007 26525 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:42:31.813970 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.791670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:35723 (local address 127.0.0.1:49318)
0806 19:42:31.802880 (+ 11210us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:31.802944 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:31.803269 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:31.806078 (+ 2809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:31.806083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:31.806133 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:31.806719 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.806734 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.808690 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.808696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.811054 (+ 2358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.811069 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.812744 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.812750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.812965 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.812988 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 19:42:31.813118 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 19:42:31.813367 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10823,"mutex_wait_us":31,"spinlock_wait_cycles":13056,"thread_start_us":13962,"threads_started":1}
I20260806 19:42:31.814023 27005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.792902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.21.202.65:42961)
0806 19:42:31.803694 (+ 10792us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:31.803708 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:31.804910 (+ 1202us) server_negotiation.cc:407] Connection header received
0806 19:42:31.805135 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:31.805146 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:31.805410 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:31.805938 (+ 528us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:31.806554 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.808233 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.811249 (+ 3016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.812416 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.812568 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.812856 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:31.813217 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 19:42:31.813396 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10560,"thread_start_us":10347,"threads_started":1}
I20260806 19:42:31.814178 27003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.792116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:42961 (local address 127.0.0.1:59750)
0806 19:42:31.804753 (+ 12637us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:31.804768 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:31.804952 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:31.805686 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:31.805698 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:31.805729 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:31.806319 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.806336 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.808389 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.808395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.811045 (+ 2650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.811060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.812654 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.812661 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.812922 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.812987 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 19:42:31.813147 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 19:42:31.813367 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12524,"mutex_wait_us":142,"thread_start_us":12350,"threads_started":1}
I20260806 19:42:31.814999 27001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.792533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:34803 (local address 127.0.0.1:54368)
0806 19:42:31.805943 (+ 13410us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:31.805969 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:31.806137 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:31.807179 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:31.807183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:31.807208 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:31.807681 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.807692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.809661 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.809670 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.811885 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.811900 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.814029 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.814041 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.814235 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.814269 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 19:42:31.814378 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 19:42:31.814531 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13260,"mutex_wait_us":72,"spinlock_wait_cycles":768,"thread_start_us":10976,"threads_started":1}
I20260806 19:42:31.815161 27006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.793319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54368 (local address 127.21.202.67:34803)
0806 19:42:31.801504 (+ 8185us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:31.801518 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:31.806309 (+ 4791us) server_negotiation.cc:407] Connection header received
0806 19:42:31.806434 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:31.806444 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:31.806690 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:31.806889 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:31.807904 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.809498 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.812069 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.813500 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.813630 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.813845 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:31.814445 (+ 600us) server_negotiation.cc:299] Negotiation successful
0806 19:42:31.814776 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7962,"thread_start_us":7774,"threads_started":1}
I20260806 19:42:31.816133 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.791929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49318 (local address 127.21.202.66:35723)
0806 19:42:31.805478 (+ 13549us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:31.805492 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:31.805524 (+ 32us) server_negotiation.cc:407] Connection header received
0806 19:42:31.805623 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:31.805630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:31.805842 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:31.806934 (+ 1092us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:31.807165 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.808563 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.811243 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.812489 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.815483 (+ 2994us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.815656 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:31.815707 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 19:42:31.815827 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13367,"thread_start_us":13268,"threads_started":1}
I20260806 19:42:31.818783 26923 tablet_service.cc:1467] Processing CreateTablet for tablet 98319754476348b2af0fe09478126474 (DEFAULT_TABLE table=test-table [id=0b7083b37d4448adab3c1e8911e047e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:31.818822 26651 tablet_service.cc:1467] Processing CreateTablet for tablet 98319754476348b2af0fe09478126474 (DEFAULT_TABLE table=test-table [id=0b7083b37d4448adab3c1e8911e047e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:31.819442 26788 tablet_service.cc:1467] Processing CreateTablet for tablet 98319754476348b2af0fe09478126474 (DEFAULT_TABLE table=test-table [id=0b7083b37d4448adab3c1e8911e047e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:31.820159 26923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98319754476348b2af0fe09478126474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:31.820412 26651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98319754476348b2af0fe09478126474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:31.820883 26788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98319754476348b2af0fe09478126474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:31.833205 27007 tablet_bootstrap.cc:492] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099: Bootstrap starting.
I20260806 19:42:31.836002 27008 tablet_bootstrap.cc:492] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264: Bootstrap starting.
I20260806 19:42:31.837641 27009 tablet_bootstrap.cc:492] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced: Bootstrap starting.
I20260806 19:42:31.838567 27007 tablet_bootstrap.cc:654] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:31.839999 27007 log.cc:824] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:31.840428 27008 tablet_bootstrap.cc:654] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:31.841537 27009 tablet_bootstrap.cc:654] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:31.841686 27008 log.cc:824] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:31.843006 27009 log.cc:824] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:31.849406 27008 tablet_bootstrap.cc:492] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264: No bootstrap required, opened a new log
I20260806 19:42:31.849591 27007 tablet_bootstrap.cc:492] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099: No bootstrap required, opened a new log
I20260806 19:42:31.849620 27009 tablet_bootstrap.cc:492] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced: No bootstrap required, opened a new log
I20260806 19:42:31.849774 27008 ts_tablet_manager.cc:1397] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.004s
I20260806 19:42:31.849886 27009 ts_tablet_manager.cc:1397] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260806 19:42:31.849886 27007 ts_tablet_manager.cc:1397] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.006s
I20260806 19:42:31.858458 27008 raft_consensus.cc:348] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893a11884d024e26a70492d59639aced" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35723 } } peers { permanent_uuid: "9c038924954a48a6a2aa054b795f1264" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 42961 } } peers { permanent_uuid: "a0d65c79c80b4119b6866390ac9fc099" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34803 } }
I20260806 19:42:31.858896 27008 raft_consensus.cc:374] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:31.859009 27008 raft_consensus.cc:729] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c038924954a48a6a2aa054b795f1264, State: Initialized, Role: FOLLOWER
I20260806 19:42:31.859678 27008 consensus_queue.cc:260] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893a11884d024e26a70492d59639aced" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35723 } } peers { permanent_uuid: "9c038924954a48a6a2aa054b795f1264" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 42961 } } peers { permanent_uuid: "a0d65c79c80b4119b6866390ac9fc099" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34803 } }
I20260806 19:42:31.861706 27008 ts_tablet_manager.cc:1428] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260806 19:42:31.861949 27007 raft_consensus.cc:348] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893a11884d024e26a70492d59639aced" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35723 } } peers { permanent_uuid: "9c038924954a48a6a2aa054b795f1264" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 42961 } } peers { permanent_uuid: "a0d65c79c80b4119b6866390ac9fc099" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34803 } }
I20260806 19:42:31.861953 27009 raft_consensus.cc:348] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893a11884d024e26a70492d59639aced" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35723 } } peers { permanent_uuid: "9c038924954a48a6a2aa054b795f1264" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 42961 } } peers { permanent_uuid: "a0d65c79c80b4119b6866390ac9fc099" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34803 } }
I20260806 19:42:31.862478 27009 raft_consensus.cc:374] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:31.862478 27007 raft_consensus.cc:374] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:31.862598 27009 raft_consensus.cc:729] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 893a11884d024e26a70492d59639aced, State: Initialized, Role: FOLLOWER
I20260806 19:42:31.862598 27007 raft_consensus.cc:729] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0d65c79c80b4119b6866390ac9fc099, State: Initialized, Role: FOLLOWER
I20260806 19:42:31.863300 27007 consensus_queue.cc:260] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893a11884d024e26a70492d59639aced" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35723 } } peers { permanent_uuid: "9c038924954a48a6a2aa054b795f1264" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 42961 } } peers { permanent_uuid: "a0d65c79c80b4119b6866390ac9fc099" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34803 } }
I20260806 19:42:31.863296 27009 consensus_queue.cc:260] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893a11884d024e26a70492d59639aced" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35723 } } peers { permanent_uuid: "9c038924954a48a6a2aa054b795f1264" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 42961 } } peers { permanent_uuid: "a0d65c79c80b4119b6866390ac9fc099" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34803 } }
I20260806 19:42:31.865640 26853 heartbeater.cc:503] Master 127.21.202.126:37487 was elected leader, sending a full tablet report...
I20260806 19:42:31.865680 26988 heartbeater.cc:503] Master 127.21.202.126:37487 was elected leader, sending a full tablet report...
I20260806 19:42:31.866393 27009 ts_tablet_manager.cc:1428] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260806 19:42:31.867501 27007 ts_tablet_manager.cc:1428] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099: Time spent starting tablet: real 0.018s user 0.014s sys 0.002s
W20260806 19:42:31.912380 26719 tablet.cc:2369] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:42:31.931087 26854 tablet.cc:2369] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:42:31.938959 27014 raft_consensus.cc:482] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:31.939425 27014 raft_consensus.cc:504] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893a11884d024e26a70492d59639aced" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35723 } } peers { permanent_uuid: "9c038924954a48a6a2aa054b795f1264" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 42961 } } peers { permanent_uuid: "a0d65c79c80b4119b6866390ac9fc099" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34803 } }
I20260806 19:42:31.941601 27014 leader_election.cc:288] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 893a11884d024e26a70492d59639aced (127.21.202.66:35723), 9c038924954a48a6a2aa054b795f1264 (127.21.202.65:42961)
I20260806 19:42:31.941964 27015 raft_consensus.cc:482] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:31.942415 27015 raft_consensus.cc:504] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893a11884d024e26a70492d59639aced" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35723 } } peers { permanent_uuid: "9c038924954a48a6a2aa054b795f1264" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 42961 } } peers { permanent_uuid: "a0d65c79c80b4119b6866390ac9fc099" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34803 } }
I20260806 19:42:31.945456 27015 leader_election.cc:288] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c038924954a48a6a2aa054b795f1264 (127.21.202.65:42961), a0d65c79c80b4119b6866390ac9fc099 (127.21.202.67:34803)
I20260806 19:42:31.950044 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.941939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:35723 (local address 127.21.202.67:46975)
0806 19:42:31.942451 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:31.942478 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:31.942637 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:31.943301 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:31.943306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:31.943348 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:31.943548 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.943562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.944890 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.944896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.946898 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.946917 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.949223 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.949229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.949442 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.949482 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 19:42:31.949576 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 19:42:31.949736 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351}
I20260806 19:42:31.950881 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.942762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:46975 (local address 127.21.202.66:35723)
0806 19:42:31.942979 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:31.942985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:31.943007 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:42:31.943082 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:31.943086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:31.943148 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:31.943267 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:31.943715 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.944687 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.947486 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.948899 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.949062 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.949133 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:31.950370 (+ 1237us) server_negotiation.cc:299] Negotiation successful
0806 19:42:31.950510 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 19:42:31.951465 27005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.942181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:46955 (local address 127.21.202.65:42961)
0806 19:42:31.942404 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:31.942414 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:31.943467 (+ 1053us) server_negotiation.cc:407] Connection header received
0806 19:42:31.943819 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:31.943825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:31.943902 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:31.944010 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:31.944716 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.945835 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.948365 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.949765 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.949928 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.949993 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:31.950977 (+ 984us) server_negotiation.cc:299] Negotiation successful
0806 19:42:31.951191 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":68}
I20260806 19:42:31.952447 27016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.942075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:42961 (local address 127.21.202.67:46955)
0806 19:42:31.943345 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:31.943365 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:31.943586 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:31.944058 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:31.944063 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:31.944091 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:31.944497 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.944511 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.946069 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.946075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.948170 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.948188 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.950440 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.950446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.950741 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.950778 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:42:31.951940 (+ 1162us) client_negotiation.cc:240] Negotiation successful
0806 19:42:31.952087 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"thread_start_us":1295,"threads_started":1}
I20260806 19:42:31.952363 26808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98319754476348b2af0fe09478126474" candidate_uuid: "a0d65c79c80b4119b6866390ac9fc099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "893a11884d024e26a70492d59639aced" is_pre_election: true
I20260806 19:42:31.953007 26808 raft_consensus.cc:2454] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0d65c79c80b4119b6866390ac9fc099 in term 0.
I20260806 19:42:31.954061 26877 leader_election.cc:302] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 893a11884d024e26a70492d59639aced, a0d65c79c80b4119b6866390ac9fc099; no voters:
I20260806 19:42:31.954222 26672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98319754476348b2af0fe09478126474" candidate_uuid: "a0d65c79c80b4119b6866390ac9fc099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c038924954a48a6a2aa054b795f1264" is_pre_election: true
I20260806 19:42:31.954560 27014 raft_consensus.cc:2781] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:42:31.954726 27014 raft_consensus.cc:482] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:31.954843 27014 raft_consensus.cc:3037] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:31.955127 26672 raft_consensus.cc:2454] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0d65c79c80b4119b6866390ac9fc099 in term 0.
I20260806 19:42:31.957472 27017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.945134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:42961 (local address 127.21.202.66:39081)
0806 19:42:31.947914 (+ 2780us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:31.947937 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:31.948104 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:31.949407 (+ 1303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:31.949416 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:31.949453 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:31.950024 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.950040 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.952065 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.952069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.954473 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.954494 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.956608 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.956615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.956792 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.956850 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 19:42:31.956931 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 19:42:31.957078 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2595,"thread_start_us":6067,"threads_started":1}
I20260806 19:42:31.958253 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.945656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:39081 (local address 127.21.202.65:42961)
0806 19:42:31.948975 (+ 3319us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:31.948984 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:31.949017 (+ 33us) server_negotiation.cc:407] Connection header received
0806 19:42:31.949094 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:31.949101 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:31.949197 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:31.950203 (+ 1006us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:31.950521 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.951907 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.954711 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.956365 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.957196 (+ 831us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.957727 (+ 531us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:31.957794 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 19:42:31.957985 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3092,"thread_start_us":2854,"threads_started":1}
I20260806 19:42:31.958499 27014 raft_consensus.cc:504] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893a11884d024e26a70492d59639aced" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35723 } } peers { permanent_uuid: "9c038924954a48a6a2aa054b795f1264" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 42961 } } peers { permanent_uuid: "a0d65c79c80b4119b6866390ac9fc099" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34803 } }
I20260806 19:42:31.958810 26672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98319754476348b2af0fe09478126474" candidate_uuid: "893a11884d024e26a70492d59639aced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c038924954a48a6a2aa054b795f1264" is_pre_election: true
I20260806 19:42:31.958915 27006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.947212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:33027 (local address 127.21.202.67:34803)
0806 19:42:31.947404 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:31.947410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:31.951725 (+ 4315us) server_negotiation.cc:407] Connection header received
0806 19:42:31.951968 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:31.951973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:31.952039 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:31.952095 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:31.953041 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.953967 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.956343 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.957767 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.957928 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.957991 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:31.958518 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 19:42:31.958653 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 19:42:31.959055 26672 raft_consensus.cc:2454] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 893a11884d024e26a70492d59639aced in term 0.
I20260806 19:42:31.959542 27014 leader_election.cc:288] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [CANDIDATE]: Term 1 election: Requested vote from peers 893a11884d024e26a70492d59639aced (127.21.202.66:35723), 9c038924954a48a6a2aa054b795f1264 (127.21.202.65:42961)
I20260806 19:42:31.959695 27018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:31.945134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:34803 (local address 127.21.202.66:33027)
0806 19:42:31.951609 (+ 6475us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:31.951630 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:31.951829 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:31.952272 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:31.952277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:31.952308 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:31.952778 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.952792 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.954167 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.954172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.956174 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:31.956186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:31.958068 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:31.958076 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:31.958303 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:31.958339 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 19:42:31.959083 (+ 744us) client_negotiation.cc:240] Negotiation successful
0806 19:42:31.959233 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6336,"mutex_wait_us":26,"thread_start_us":2317,"threads_started":1}
I20260806 19:42:31.959874 26808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98319754476348b2af0fe09478126474" candidate_uuid: "a0d65c79c80b4119b6866390ac9fc099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "893a11884d024e26a70492d59639aced"
I20260806 19:42:31.959827 26743 leader_election.cc:302] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 893a11884d024e26a70492d59639aced, 9c038924954a48a6a2aa054b795f1264; no voters:
I20260806 19:42:31.960111 26808 raft_consensus.cc:3037] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:31.960386 26672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98319754476348b2af0fe09478126474" candidate_uuid: "a0d65c79c80b4119b6866390ac9fc099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c038924954a48a6a2aa054b795f1264"
I20260806 19:42:31.960608 26672 raft_consensus.cc:3037] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264 [term 0 FOLLOWER]: Advancing to term 1
W20260806 19:42:31.960794 26989 tablet.cc:2369] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:42:31.961097 26943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98319754476348b2af0fe09478126474" candidate_uuid: "893a11884d024e26a70492d59639aced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0d65c79c80b4119b6866390ac9fc099" is_pre_election: true
I20260806 19:42:31.961736 26943 raft_consensus.cc:2379] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 893a11884d024e26a70492d59639aced in current term 1: Already voted for candidate a0d65c79c80b4119b6866390ac9fc099 in this term.
I20260806 19:42:31.963519 26808 raft_consensus.cc:2454] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0d65c79c80b4119b6866390ac9fc099 in term 1.
I20260806 19:42:31.963742 27015 raft_consensus.cc:2741] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 19:42:31.963847 26672 raft_consensus.cc:2454] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0d65c79c80b4119b6866390ac9fc099 in term 1.
I20260806 19:42:31.964022 26877 leader_election.cc:302] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 893a11884d024e26a70492d59639aced, a0d65c79c80b4119b6866390ac9fc099; no voters:
I20260806 19:42:31.964435 27014 raft_consensus.cc:2781] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:31.965006 27014 raft_consensus.cc:686] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 1 LEADER]: Becoming Leader. State: Replica: a0d65c79c80b4119b6866390ac9fc099, State: Running, Role: LEADER
I20260806 19:42:31.965524 27014 consensus_queue.cc:237] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893a11884d024e26a70492d59639aced" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35723 } } peers { permanent_uuid: "9c038924954a48a6a2aa054b795f1264" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 42961 } } peers { permanent_uuid: "a0d65c79c80b4119b6866390ac9fc099" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34803 } }
I20260806 19:42:31.970914 26521 catalog_manager.cc:5495] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0d65c79c80b4119b6866390ac9fc099 (127.21.202.67). New cstate: current_term: 1 leader_uuid: "a0d65c79c80b4119b6866390ac9fc099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893a11884d024e26a70492d59639aced" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c038924954a48a6a2aa054b795f1264" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 42961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0d65c79c80b4119b6866390ac9fc099" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34803 } health_report { overall_health: HEALTHY } } }
I20260806 19:42:32.021088 27006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:32.015295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.21.202.67:34803)
0806 19:42:32.015485 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:32.015490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:32.015571 (+ 81us) server_negotiation.cc:407] Connection header received
0806 19:42:32.015757 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:32.015762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:32.015842 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:32.015982 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:32.016328 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:32.017209 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:32.018622 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:32.019130 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:32.019255 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:32.019849 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:32.020386 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:32.020519 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:32.020667 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 19:42:32.020741 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 19:42:32.030808 26808 raft_consensus.cc:1261] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [term 1 FOLLOWER]: Refusing update from remote peer a0d65c79c80b4119b6866390ac9fc099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:32.030928 26672 raft_consensus.cc:1261] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264 [term 1 FOLLOWER]: Refusing update from remote peer a0d65c79c80b4119b6866390ac9fc099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:32.031875 27020 consensus_queue.cc:1035] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c038924954a48a6a2aa054b795f1264" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 42961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:32.032336 27014 consensus_queue.cc:1035] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "893a11884d024e26a70492d59639aced" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:32.044288 27024 mvcc.cc:205] Tried to move back new op lower bound from 7315641761907359744 to 7315641761654616064. Current Snapshot: MvccSnapshot[applied={T|T < 7315641761907359744}]
I20260806 19:42:32.064864 26525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:32.065192 26525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:32.069634 26525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10d2519f5b7045588fa255d92facdb06: Sending DeleteTablet for 3 replicas of tablet 98319754476348b2af0fe09478126474
I20260806 19:42:32.071113 26923 tablet_service.cc:1514] Processing DeleteTablet for tablet 98319754476348b2af0fe09478126474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:32 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:54368
I20260806 19:42:32.071165 26651 tablet_service.cc:1514] Processing DeleteTablet for tablet 98319754476348b2af0fe09478126474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:32 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:59750
I20260806 19:42:32.071300 26788 tablet_service.cc:1514] Processing DeleteTablet for tablet 98319754476348b2af0fe09478126474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:32 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:49318
I20260806 19:42:32.073520 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 26587
I20260806 19:42:32.083221 27034 tablet_replica.cc:330] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264: stopping tablet replica
I20260806 19:42:32.083920 27034 raft_consensus.cc:2229] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:32.086853 27034 raft_consensus.cc:2258] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:32.092485 27035 tablet_replica.cc:330] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099: stopping tablet replica
I20260806 19:42:32.093139 27035 raft_consensus.cc:2229] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:42:32.094043 27035 raft_consensus.cc:2258] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:32.089924 27034 ts_tablet_manager.cc:1905] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:32.095005 27036 tablet_replica.cc:330] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced: stopping tablet replica
I20260806 19:42:32.095503 27036 raft_consensus.cc:2229] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:32.096058 27036 raft_consensus.cc:2258] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 06 19:42:32 dist-test-slave-77v9 krb5kdc[26479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:42:32.098632 27036 ts_tablet_manager.cc:1905] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:32.102594 27034 ts_tablet_manager.cc:1918] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:32.102797 27034 log.cc:1196] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/98319754476348b2af0fe09478126474
I20260806 19:42:32.103394 27034 ts_tablet_manager.cc:1939] T 98319754476348b2af0fe09478126474 P 9c038924954a48a6a2aa054b795f1264: Deleting consensus metadata
I20260806 19:42:32.105445 26510 catalog_manager.cc:4841] TS 9c038924954a48a6a2aa054b795f1264 (127.21.202.65:42961): tablet 98319754476348b2af0fe09478126474 (table test-table [id=0b7083b37d4448adab3c1e8911e047e9]) successfully deleted
I20260806 19:42:32.106269 27036 ts_tablet_manager.cc:1918] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:32.106492 27036 log.cc:1196] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/98319754476348b2af0fe09478126474
I20260806 19:42:32.107076 27036 ts_tablet_manager.cc:1939] T 98319754476348b2af0fe09478126474 P 893a11884d024e26a70492d59639aced: Deleting consensus metadata
I20260806 19:42:32.107856 27035 ts_tablet_manager.cc:1905] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:32.108844 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:32.075273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.21.202.65:42961)
0806 19:42:32.082967 (+ 7694us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:32.082975 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:32.083004 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:42:32.083113 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:32.083120 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:32.083226 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:32.083481 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:32.085708 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:32.087076 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:32.089729 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:32.090569 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:32.090688 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:32.098437 (+ 7749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:32.098456 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:32.098477 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:32.098535 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:32.103659 (+ 5124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:32.104728 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:32.104739 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:32.104743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:32.104974 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:32.105464 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:32.105469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:32.105474 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:32.106049 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:32.107009 (+ 960us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:32.107685 (+ 676us) server_negotiation.cc:299] Negotiation successful
0806 19:42:32.107981 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7541}
I20260806 19:42:32.108982 26509 catalog_manager.cc:4841] TS 893a11884d024e26a70492d59639aced (127.21.202.66:35723): tablet 98319754476348b2af0fe09478126474 (table test-table [id=0b7083b37d4448adab3c1e8911e047e9]) successfully deleted
I20260806 19:42:32.110730 26713 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:32.115955 27035 ts_tablet_manager.cc:1918] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:32.116156 27035 log.cc:1196] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/98319754476348b2af0fe09478126474
I20260806 19:42:32.116736 27035 ts_tablet_manager.cc:1939] T 98319754476348b2af0fe09478126474 P a0d65c79c80b4119b6866390ac9fc099: Deleting consensus metadata
I20260806 19:42:32.118868 26507 catalog_manager.cc:4841] TS a0d65c79c80b4119b6866390ac9fc099 (127.21.202.67:34803): tablet 98319754476348b2af0fe09478126474 (table test-table [id=0b7083b37d4448adab3c1e8911e047e9]) successfully deleted
I20260806 19:42:32.205659 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 26587
I20260806 19:42:32.226434 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 26724
Aug 06 19:42:32 dist-test-slave-77v9 krb5kdc[26479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:42:32.244343 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:32.227372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49334 (local address 127.21.202.66:35723)
0806 19:42:32.227573 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:32.227579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:32.227601 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:42:32.227672 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:32.227678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:32.227764 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:32.227883 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:32.229148 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:32.230351 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:32.231557 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:32.232200 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:32.232347 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:32.236983 (+ 4636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:32.237003 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:32.237026 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:32.237077 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:32.241132 (+ 4055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:32.241979 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:32.241991 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:32.241995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:32.242159 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:32.242561 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:32.242566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:32.242570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:32.243088 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:32.243432 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:32.243672 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 19:42:32.243921 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 19:42:32.245204 26848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:32.340637 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 26724
I20260806 19:42:32.360141 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 26858
Aug 06 19:42:32 dist-test-slave-77v9 krb5kdc[26479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:42:32.378896 27006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:32.361060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54382 (local address 127.21.202.67:34803)
0806 19:42:32.361296 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:32.361301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:32.361417 (+ 116us) server_negotiation.cc:407] Connection header received
0806 19:42:32.361608 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:32.361612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:32.361676 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:32.361758 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:32.363548 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:32.364278 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:32.366338 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:32.366816 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:32.366916 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:32.372475 (+ 5559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:32.372490 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:32.372507 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:32.372549 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:32.375882 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:32.376617 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:32.376625 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:32.376627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:32.376767 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:32.377297 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:32.377301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:32.377304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:32.377669 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:32.378021 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:32.378316 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 19:42:32.378576 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 19:42:32.379580 26983 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:32.504771 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 26858
I20260806 19:42:32.527947 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 26492
I20260806 19:42:32.529315 26555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:32.641636 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 26492
2026-08-06T19:42:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4508 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T19:42:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:32Z Disabled control of system clock
I20260806 19:42:32.722760 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:40463
--webserver_interface=127.21.202.126
--webserver_port=0
--builtin_ntp_servers=127.21.202.84:37437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:40463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 19:42:33.057461 27046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:33.057778 27046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:33.057933 27046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:33.065164 27046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:33.065268 27046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:33.065306 27046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:33.065431 27046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:33.065516 27046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:42:33.074779 27046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:40463
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:40463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:33.076202 27046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:33.078037 27046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:33.090585 27051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:33.090512 27052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:33.090559 27054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:33.091501 27046 server_base.cc:1034] running on GCE node
I20260806 19:42:33.092373 27046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:33.094030 27046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:33.095254 27046 hybrid_clock.cc:648] HybridClock initialized: now 1786045353095211 us; error 40 us; skew 500 ppm
I20260806 19:42:33.095654 27046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:42:33.098358 27046 webserver.cc:460] Webserver started at http://127.21.202.126:34081/ using document root <none> and password file <none>
I20260806 19:42:33.099094 27046 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:33.099184 27046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:33.099542 27046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:33.102618 27046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "ca3f16e5f57a4611a74acea8a5e22d74"
format_stamp: "Formatted at 2026-08-06 19:42:33 on dist-test-slave-77v9"
server_key: "c7cf27b573e6fe3ee30f959de6997842"
server_key_iv: "3c88b12a3d05229dcee6998bb0a55b4a"
server_key_version: "encryptionkey@0"
I20260806 19:42:33.103452 27046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "ca3f16e5f57a4611a74acea8a5e22d74"
format_stamp: "Formatted at 2026-08-06 19:42:33 on dist-test-slave-77v9"
server_key: "c7cf27b573e6fe3ee30f959de6997842"
server_key_iv: "3c88b12a3d05229dcee6998bb0a55b4a"
server_key_version: "encryptionkey@0"
I20260806 19:42:33.108731 27046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 19:42:33.112300 27061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:33.113758 27046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:33.114081 27046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "ca3f16e5f57a4611a74acea8a5e22d74"
format_stamp: "Formatted at 2026-08-06 19:42:33 on dist-test-slave-77v9"
server_key: "c7cf27b573e6fe3ee30f959de6997842"
server_key_iv: "3c88b12a3d05229dcee6998bb0a55b4a"
server_key_version: "encryptionkey@0"
I20260806 19:42:33.114351 27046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:33.120422 27046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:33.121300 27046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:33.121611 27046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:33.141104 27046 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:40463
I20260806 19:42:33.141146 27112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:40463 every 8 connection(s)
I20260806 19:42:33.142431 27046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:33.146133 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:33.146324 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 27046
I20260806 19:42:33.146574 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:33.147321 22313 external_mini_cluster.cc:1428] Setting key ede50d9f59ccd414c925bfb7ccb35268
I20260806 19:42:33.156451 27116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:33.149606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.21.202.126:40463)
0806 19:42:33.150852 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:33.150864 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:33.150910 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:42:33.152592 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:33.152616 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:33.153140 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:33.153513 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:33.154754 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:33.154761 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:33.154776 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:33.154795 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:33.154866 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:33.154884 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:33.155071 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:33.155177 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 19:42:33.155456 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":498,"threads_started":1}
I20260806 19:42:33.160090 27113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74: Bootstrap starting.
I20260806 19:42:33.163264 27113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:33.164268 27113 log.cc:824] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:33.167346 27113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74: No bootstrap required, opened a new log
I20260806 19:42:33.174523 27113 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca3f16e5f57a4611a74acea8a5e22d74" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40463 } }
I20260806 19:42:33.174909 27113 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:33.175009 27113 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca3f16e5f57a4611a74acea8a5e22d74, State: Initialized, Role: FOLLOWER
I20260806 19:42:33.175576 27113 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca3f16e5f57a4611a74acea8a5e22d74" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40463 } }
I20260806 19:42:33.175822 27113 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:33.175987 27113 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:33.176167 27113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:33.178396 27113 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca3f16e5f57a4611a74acea8a5e22d74" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40463 } }
I20260806 19:42:33.178903 27113 leader_election.cc:302] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca3f16e5f57a4611a74acea8a5e22d74; no voters:
I20260806 19:42:33.179529 27113 leader_election.cc:288] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:33.179679 27118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:33.181164 27118 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [term 1 LEADER]: Becoming Leader. State: Replica: ca3f16e5f57a4611a74acea8a5e22d74, State: Running, Role: LEADER
I20260806 19:42:33.181716 27118 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca3f16e5f57a4611a74acea8a5e22d74" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40463 } }
I20260806 19:42:33.182039 27113 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:33.184428 27120 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca3f16e5f57a4611a74acea8a5e22d74. Latest consensus state: current_term: 1 leader_uuid: "ca3f16e5f57a4611a74acea8a5e22d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca3f16e5f57a4611a74acea8a5e22d74" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40463 } } }
I20260806 19:42:33.185031 27120 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:33.185384 27119 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca3f16e5f57a4611a74acea8a5e22d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca3f16e5f57a4611a74acea8a5e22d74" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40463 } } }
I20260806 19:42:33.185946 27119 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:33.187801 27128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:33.193230 27128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:33.202850 27128 catalog_manager.cc:1347] Generated new cluster ID: 0d0d269d8c2648da9895b3bcf003878a
I20260806 19:42:33.202967 27128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:33.232842 27128 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:33.233343 27128 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:33.243003 27128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74: Generated new TSK 0
I20260806 19:42:33.243981 27128 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:42:33.257938 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:40463
--builtin_ntp_servers=127.21.202.84:37437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 19:42:33.616745 27137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:33.617107 27137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:33.617285 27137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:33.624994 27137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:33.625137 27137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:33.625346 27137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:42:33.635269 27137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:40463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:33.636844 27137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:33.638890 27137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:33.653190 27142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:33.653275 27143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:33.653208 27145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:33.655305 27137 server_base.cc:1034] running on GCE node
I20260806 19:42:33.656297 27137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:33.657493 27137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:33.658722 27137 hybrid_clock.cc:648] HybridClock initialized: now 1786045353658679 us; error 34 us; skew 500 ppm
I20260806 19:42:33.659094 27137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:42:33.661912 27137 webserver.cc:460] Webserver started at http://127.21.202.65:34169/ using document root <none> and password file <none>
I20260806 19:42:33.662694 27137 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:33.662783 27137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:33.663074 27137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:33.666103 27137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "ac176c40c94646168bd577919dac1ac4"
format_stamp: "Formatted at 2026-08-06 19:42:33 on dist-test-slave-77v9"
server_key: "36aa255edafa0c66e8ace46ea5d6e401"
server_key_iv: "fc58b56582d873aab0b0765a768393cb"
server_key_version: "encryptionkey@0"
I20260806 19:42:33.666893 27137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "ac176c40c94646168bd577919dac1ac4"
format_stamp: "Formatted at 2026-08-06 19:42:33 on dist-test-slave-77v9"
server_key: "36aa255edafa0c66e8ace46ea5d6e401"
server_key_iv: "fc58b56582d873aab0b0765a768393cb"
server_key_version: "encryptionkey@0"
I20260806 19:42:33.672387 27137 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:42:33.676709 27151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:33.678260 27137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:42:33.678457 27137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "ac176c40c94646168bd577919dac1ac4"
format_stamp: "Formatted at 2026-08-06 19:42:33 on dist-test-slave-77v9"
server_key: "36aa255edafa0c66e8ace46ea5d6e401"
server_key_iv: "fc58b56582d873aab0b0765a768393cb"
server_key_version: "encryptionkey@0"
I20260806 19:42:33.678686 27137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:33.684938 27137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:33.685781 27137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:33.686061 27137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:33.687269 27137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:33.688870 27137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:33.688987 27137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:33.689121 27137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:33.689205 27137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:33.725405 27137 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:39541
I20260806 19:42:33.725497 27265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:39541 every 8 connection(s)
I20260806 19:42:33.726787 27137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:42:33.733943 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 27137
I20260806 19:42:33.734179 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:42:33.734829 22313 external_mini_cluster.cc:1428] Setting key 1c800f74f0d0264cc286ce448ffcce2b
I20260806 19:42:33.735759 27269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:33.729535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:56479 (local address 127.21.202.126:40463)
0806 19:42:33.730001 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:33.730005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:33.731280 (+ 1275us) server_negotiation.cc:407] Connection header received
0806 19:42:33.733024 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:33.733029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:33.733085 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:33.733191 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:33.734869 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:33.734874 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:33.734877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:33.734895 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:33.734965 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:33.734984 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:33.735123 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:33.735331 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 19:42:33.735408 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":233,"threads_started":1}
I20260806 19:42:33.736680 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:33.730196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:40463 (local address 127.21.202.65:56479)
0806 19:42:33.731105 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:33.731149 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:33.732641 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:33.733392 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:33.733443 (+ 51us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:33.733560 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:33.734511 (+ 951us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:42:33.734514 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:33.734552 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:42:33.734639 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:33.735095 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:33.735108 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:33.735116 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 19:42:33.735424 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 19:42:33.735688 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":275,"threads_started":1}
I20260806 19:42:33.737777 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:40463
--builtin_ntp_servers=127.21.202.84:37437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 19:42:33.738370 27266 heartbeater.cc:347] Connected to a master server at 127.21.202.126:40463
I20260806 19:42:33.738703 27266 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:33.739441 27266 heartbeater.cc:511] Master 127.21.202.126:40463 requested a full tablet report, sending...
I20260806 19:42:33.741935 27078 ts_manager.cc:194] Registered new tserver with Master: ac176c40c94646168bd577919dac1ac4 (127.21.202.65:39541)
W20260806 19:42:34.070026 27271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:34.070324 27271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:34.070417 27271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:34.077757 27271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:34.077883 27271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:34.077996 27271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:42:34.087693 27271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:40463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:34.089105 27271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:34.090873 27271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:34.104385 27276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:34.104427 27277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:34.104473 27279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:34.106400 27271 server_base.cc:1034] running on GCE node
I20260806 19:42:34.107069 27271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:34.108119 27271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:34.109364 27271 hybrid_clock.cc:648] HybridClock initialized: now 1786045354109303 us; error 57 us; skew 500 ppm
I20260806 19:42:34.109761 27271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:42:34.112413 27271 webserver.cc:460] Webserver started at http://127.21.202.66:36055/ using document root <none> and password file <none>
I20260806 19:42:34.113328 27271 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:34.113437 27271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:34.113847 27271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:34.117165 27271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "287da06b2e6a44b3a1be9eb4a68f8319"
format_stamp: "Formatted at 2026-08-06 19:42:34 on dist-test-slave-77v9"
server_key: "9081a79d700bfa90713905c84c727ac8"
server_key_iv: "ac3f0149e3eafbca32f22ec46a4f5f4c"
server_key_version: "encryptionkey@0"
I20260806 19:42:34.118120 27271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "287da06b2e6a44b3a1be9eb4a68f8319"
format_stamp: "Formatted at 2026-08-06 19:42:34 on dist-test-slave-77v9"
server_key: "9081a79d700bfa90713905c84c727ac8"
server_key_iv: "ac3f0149e3eafbca32f22ec46a4f5f4c"
server_key_version: "encryptionkey@0"
I20260806 19:42:34.124070 27271 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 19:42:34.128090 27285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:34.129642 27271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:34.129837 27271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "287da06b2e6a44b3a1be9eb4a68f8319"
format_stamp: "Formatted at 2026-08-06 19:42:34 on dist-test-slave-77v9"
server_key: "9081a79d700bfa90713905c84c727ac8"
server_key_iv: "ac3f0149e3eafbca32f22ec46a4f5f4c"
server_key_version: "encryptionkey@0"
I20260806 19:42:34.130085 27271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:34.136183 27271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:34.137090 27271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:34.137369 27271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:34.138294 27271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:34.139775 27271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:34.139863 27271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:34.140020 27271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:34.140097 27271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:34.175197 27271 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:35281
I20260806 19:42:34.175290 27399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:35281 every 8 connection(s)
I20260806 19:42:34.176586 27271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:42:34.182394 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 27271
I20260806 19:42:34.182610 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:42:34.183275 22313 external_mini_cluster.cc:1428] Setting key baab8db75a21d0ba5b132fe2665850e2
I20260806 19:42:34.186741 27269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:34.179443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:42955 (local address 127.21.202.126:40463)
0806 19:42:34.179678 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:34.179685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:34.181189 (+ 1504us) server_negotiation.cc:407] Connection header received
0806 19:42:34.183918 (+ 2729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:34.183923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:34.184026 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:34.184157 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:34.185910 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:34.185914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:34.185917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:34.185932 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:34.185963 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:34.185975 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:34.186068 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:34.186356 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 19:42:34.186427 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 19:42:34.186995 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:40463
--builtin_ntp_servers=127.21.202.84:37437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 19:42:34.188115 27402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:34.179936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:40463 (local address 127.21.202.66:42955)
0806 19:42:34.181019 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:34.181065 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:34.183520 (+ 2455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:34.184526 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:34.184541 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:34.184667 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:34.185543 (+ 876us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:42:34.185549 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:34.185602 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:42:34.185725 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:34.186083 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:34.186095 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:34.186102 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 19:42:34.186419 (+ 317us) client_negotiation.cc:240] Negotiation successful
0806 19:42:34.186773 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":338,"threads_started":1}
I20260806 19:42:34.189606 27400 heartbeater.cc:347] Connected to a master server at 127.21.202.126:40463
I20260806 19:42:34.189929 27400 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:34.190536 27400 heartbeater.cc:511] Master 127.21.202.126:40463 requested a full tablet report, sending...
I20260806 19:42:34.191897 27078 ts_manager.cc:194] Registered new tserver with Master: 287da06b2e6a44b3a1be9eb4a68f8319 (127.21.202.66:35281)
W20260806 19:42:34.522524 27403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:34.522861 27403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:34.523034 27403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:34.530436 27403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:34.530578 27403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:34.530763 27403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:42:34.540182 27403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:40463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:34.541678 27403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:34.543531 27403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:34.557174 27408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:34.557663 27411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:34.558306 27409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:34.558738 27403 server_base.cc:1034] running on GCE node
I20260806 19:42:34.559324 27403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:34.560350 27403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:34.561576 27403 hybrid_clock.cc:648] HybridClock initialized: now 1786045354561530 us; error 30 us; skew 500 ppm
I20260806 19:42:34.561894 27403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:42:34.565521 27403 webserver.cc:460] Webserver started at http://127.21.202.67:36089/ using document root <none> and password file <none>
I20260806 19:42:34.566206 27403 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:34.566294 27403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:34.566598 27403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:34.569526 27403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "701da40e5e94457b8b04980436e43b41"
format_stamp: "Formatted at 2026-08-06 19:42:34 on dist-test-slave-77v9"
server_key: "3826c4788dcb01fe8495c0ad70f7f9c9"
server_key_iv: "cad699e6eaa24a1de1bdd9fa30dc7e91"
server_key_version: "encryptionkey@0"
I20260806 19:42:34.570262 27403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "701da40e5e94457b8b04980436e43b41"
format_stamp: "Formatted at 2026-08-06 19:42:34 on dist-test-slave-77v9"
server_key: "3826c4788dcb01fe8495c0ad70f7f9c9"
server_key_iv: "cad699e6eaa24a1de1bdd9fa30dc7e91"
server_key_version: "encryptionkey@0"
I20260806 19:42:34.575474 27403 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:42:34.579330 27417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:34.580936 27403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 19:42:34.581125 27403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "701da40e5e94457b8b04980436e43b41"
format_stamp: "Formatted at 2026-08-06 19:42:34 on dist-test-slave-77v9"
server_key: "3826c4788dcb01fe8495c0ad70f7f9c9"
server_key_iv: "cad699e6eaa24a1de1bdd9fa30dc7e91"
server_key_version: "encryptionkey@0"
I20260806 19:42:34.581362 27403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:34.588445 27403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:34.589418 27403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:34.589720 27403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:34.590745 27403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:34.592309 27403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:34.592437 27403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:34.592590 27403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:34.592675 27403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:34.629335 27403 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:42829
I20260806 19:42:34.629408 27530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:42829 every 8 connection(s)
I20260806 19:42:34.630692 27403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:42:34.631913 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 27403
I20260806 19:42:34.632109 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:42:34.632687 22313 external_mini_cluster.cc:1428] Setting key 120cee52a7e12bd4aebfea875addd3e3
I20260806 19:42:34.639768 27269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:34.633815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:49857 (local address 127.21.202.126:40463)
0806 19:42:34.634030 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:34.634036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:34.635744 (+ 1708us) server_negotiation.cc:407] Connection header received
0806 19:42:34.637535 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:34.637541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:34.637608 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:34.637703 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:34.638961 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:34.638966 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:34.638971 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:34.639035 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:34.639068 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:34.639080 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:34.639201 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:34.639416 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 19:42:34.639488 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 19:42:34.640652 27533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:34.634499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:40463 (local address 127.21.202.67:49857)
0806 19:42:34.635596 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:34.635640 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:34.637275 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:34.637935 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:34.637943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:34.638029 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:34.638655 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:42:34.638657 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:34.638693 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:42:34.638775 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:34.639176 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:34.639188 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:34.639196 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 19:42:34.639497 (+ 301us) client_negotiation.cc:240] Negotiation successful
0806 19:42:34.639740 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":378,"threads_started":1}
I20260806 19:42:34.641965 27531 heartbeater.cc:347] Connected to a master server at 127.21.202.126:40463
I20260806 19:42:34.642215 27531 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:34.642750 27531 heartbeater.cc:511] Master 127.21.202.126:40463 requested a full tablet report, sending...
I20260806 19:42:34.644141 27078 ts_manager.cc:194] Registered new tserver with Master: 701da40e5e94457b8b04980436e43b41 (127.21.202.67:42829)
I20260806 19:42:34.648664 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:42:34.656435 27269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:34.653633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35888 (local address 127.21.202.126:40463)
0806 19:42:34.653806 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:34.653811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:34.654444 (+ 633us) server_negotiation.cc:407] Connection header received
0806 19:42:34.654629 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:34.654633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:34.654689 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:34.654782 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:34.655765 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:34.655769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:34.655773 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:34.655821 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:34.655854 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:34.655867 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:34.655994 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:34.656142 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 19:42:34.656194 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 19:42:34.660609 27078 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:42:34.667534 27078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:42:34.687799 27544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:34.681256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:42829 (local address 127.0.0.1:41418)
0806 19:42:34.682484 (+ 1228us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:34.682521 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:34.682682 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:34.683942 (+ 1260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:34.683953 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:34.684052 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:34.685099 (+ 1047us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:42:34.685103 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:34.685133 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:42:34.685228 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:34.686921 (+ 1693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:34.686935 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:34.686945 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 19:42:34.687133 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 19:42:34.687235 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":995,"thread_start_us":778,"threads_started":1}
I20260806 19:42:34.688329 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:34.684211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43518 (local address 127.21.202.65:39541)
0806 19:42:34.685143 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:34.685151 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:34.685176 (+ 25us) server_negotiation.cc:407] Connection header received
0806 19:42:34.685249 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:34.685256 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:34.685432 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:34.686126 (+ 694us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:34.687518 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:34.687527 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:34.687539 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:34.687555 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:34.687607 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:34.687631 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:34.687766 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:34.687875 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 19:42:34.687986 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":1785,"threads_started":1}
I20260806 19:42:34.688467 27547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:34.683652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:39541 (local address 127.0.0.1:43518)
0806 19:42:34.684576 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:34.684594 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:34.684673 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:34.685635 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:34.685639 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:34.685661 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:34.686381 (+ 720us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:42:34.686384 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:34.686401 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:42:34.686478 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:34.687729 (+ 1251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:34.687734 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:34.687738 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 19:42:34.687914 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 19:42:34.688012 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":738,"threads_started":1}
I20260806 19:42:34.689036 27545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:34.681841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38370 (local address 127.21.202.66:35281)
0806 19:42:34.684218 (+ 2377us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:34.684228 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:34.684957 (+ 729us) server_negotiation.cc:407] Connection header received
0806 19:42:34.685043 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:34.685050 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:34.685318 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:34.685461 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:34.687378 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:34.687388 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:34.687403 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:34.687431 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:34.687482 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:34.687501 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:34.687671 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:34.688033 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 19:42:34.688122 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2200,"thread_start_us":2052,"threads_started":1}
I20260806 19:42:34.689857 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:34.681249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41418 (local address 127.21.202.67:42829)
0806 19:42:34.683309 (+ 2060us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:34.683319 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:34.683357 (+ 38us) server_negotiation.cc:407] Connection header received
0806 19:42:34.683433 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:34.683444 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:34.683692 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:34.685408 (+ 1716us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:34.686621 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:34.686631 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:34.686646 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:34.686667 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:34.686733 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:34.686755 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:34.689082 (+ 2327us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:34.689356 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 19:42:34.689447 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1870,"thread_start_us":1700,"threads_started":1}
I20260806 19:42:34.689944 27546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:34.681260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:35281 (local address 127.0.0.1:38370)
0806 19:42:34.684280 (+ 3020us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:34.684306 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:34.684439 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:34.685832 (+ 1393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:34.685838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:34.685861 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:34.687117 (+ 1256us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:42:34.687121 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:34.687141 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:42:34.687189 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:34.687746 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:34.687752 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:34.687760 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 19:42:34.689368 (+ 1608us) client_negotiation.cc:240] Negotiation successful
0806 19:42:34.689421 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2308,"mutex_wait_us":783,"thread_start_us":1351,"threads_started":1}
I20260806 19:42:34.692303 27200 tablet_service.cc:1467] Processing CreateTablet for tablet 86667cc78ed44b07aa357dfd2db18928 (DEFAULT_TABLE table=test-table [id=7961c9e265254779970ede9bdccb6a65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:34.693797 27200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86667cc78ed44b07aa357dfd2db18928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:34.693306 27466 tablet_service.cc:1467] Processing CreateTablet for tablet 86667cc78ed44b07aa357dfd2db18928 (DEFAULT_TABLE table=test-table [id=7961c9e265254779970ede9bdccb6a65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:34.693306 27333 tablet_service.cc:1467] Processing CreateTablet for tablet 86667cc78ed44b07aa357dfd2db18928 (DEFAULT_TABLE table=test-table [id=7961c9e265254779970ede9bdccb6a65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:34.694645 27466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86667cc78ed44b07aa357dfd2db18928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:34.694648 27333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86667cc78ed44b07aa357dfd2db18928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:34.706420 27549 tablet_bootstrap.cc:492] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4: Bootstrap starting.
I20260806 19:42:34.710260 27549 tablet_bootstrap.cc:654] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:34.710260 27550 tablet_bootstrap.cc:492] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319: Bootstrap starting.
I20260806 19:42:34.711315 27552 tablet_bootstrap.cc:492] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41: Bootstrap starting.
I20260806 19:42:34.713824 27549 log.cc:824] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:34.714401 27550 tablet_bootstrap.cc:654] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:34.714776 27552 tablet_bootstrap.cc:654] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:34.715764 27550 log.cc:824] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:34.715934 27552 log.cc:824] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:34.717747 27549 tablet_bootstrap.cc:492] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4: No bootstrap required, opened a new log
I20260806 19:42:34.718075 27549 ts_tablet_manager.cc:1397] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.001s
I20260806 19:42:34.718948 27550 tablet_bootstrap.cc:492] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319: No bootstrap required, opened a new log
I20260806 19:42:34.719214 27550 ts_tablet_manager.cc:1397] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 19:42:34.719302 27552 tablet_bootstrap.cc:492] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41: No bootstrap required, opened a new log
I20260806 19:42:34.719547 27552 ts_tablet_manager.cc:1397] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 19:42:34.728879 27549 raft_consensus.cc:348] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "701da40e5e94457b8b04980436e43b41" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42829 } } peers { permanent_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35281 } } peers { permanent_uuid: "ac176c40c94646168bd577919dac1ac4" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39541 } }
I20260806 19:42:34.729468 27549 raft_consensus.cc:374] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:34.729569 27549 raft_consensus.cc:729] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac176c40c94646168bd577919dac1ac4, State: Initialized, Role: FOLLOWER
I20260806 19:42:34.730422 27549 consensus_queue.cc:260] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "701da40e5e94457b8b04980436e43b41" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42829 } } peers { permanent_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35281 } } peers { permanent_uuid: "ac176c40c94646168bd577919dac1ac4" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39541 } }
I20260806 19:42:34.730437 27550 raft_consensus.cc:348] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "701da40e5e94457b8b04980436e43b41" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42829 } } peers { permanent_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35281 } } peers { permanent_uuid: "ac176c40c94646168bd577919dac1ac4" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39541 } }
I20260806 19:42:34.730435 27552 raft_consensus.cc:348] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "701da40e5e94457b8b04980436e43b41" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42829 } } peers { permanent_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35281 } } peers { permanent_uuid: "ac176c40c94646168bd577919dac1ac4" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39541 } }
I20260806 19:42:34.730880 27550 raft_consensus.cc:374] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:34.730953 27552 raft_consensus.cc:374] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:34.730983 27550 raft_consensus.cc:729] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 287da06b2e6a44b3a1be9eb4a68f8319, State: Initialized, Role: FOLLOWER
I20260806 19:42:34.731045 27552 raft_consensus.cc:729] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 701da40e5e94457b8b04980436e43b41, State: Initialized, Role: FOLLOWER
I20260806 19:42:34.731678 27550 consensus_queue.cc:260] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "701da40e5e94457b8b04980436e43b41" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42829 } } peers { permanent_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35281 } } peers { permanent_uuid: "ac176c40c94646168bd577919dac1ac4" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39541 } }
I20260806 19:42:34.731666 27552 consensus_queue.cc:260] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "701da40e5e94457b8b04980436e43b41" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42829 } } peers { permanent_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35281 } } peers { permanent_uuid: "ac176c40c94646168bd577919dac1ac4" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39541 } }
I20260806 19:42:34.738935 27549 ts_tablet_manager.cc:1428] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4: Time spent starting tablet: real 0.021s user 0.007s sys 0.013s
I20260806 19:42:34.739387 27266 heartbeater.cc:503] Master 127.21.202.126:40463 was elected leader, sending a full tablet report...
I20260806 19:42:34.747314 27531 heartbeater.cc:503] Master 127.21.202.126:40463 was elected leader, sending a full tablet report...
I20260806 19:42:34.747634 27550 ts_tablet_manager.cc:1428] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319: Time spent starting tablet: real 0.028s user 0.014s sys 0.013s
I20260806 19:42:34.747892 27552 ts_tablet_manager.cc:1428] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41: Time spent starting tablet: real 0.028s user 0.020s sys 0.009s
I20260806 19:42:34.748134 27400 heartbeater.cc:503] Master 127.21.202.126:40463 was elected leader, sending a full tablet report...
W20260806 19:42:34.882530 27532 tablet.cc:2369] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:42:34.928951 27401 tablet.cc:2369] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:42:34.979532 27268 tablet.cc:2369] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:42:35.100317 27556 raft_consensus.cc:482] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:35.100682 27556 raft_consensus.cc:504] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "701da40e5e94457b8b04980436e43b41" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42829 } } peers { permanent_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35281 } } peers { permanent_uuid: "ac176c40c94646168bd577919dac1ac4" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39541 } }
I20260806 19:42:35.102742 27556 leader_election.cc:288] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 701da40e5e94457b8b04980436e43b41 (127.21.202.67:42829), 287da06b2e6a44b3a1be9eb4a68f8319 (127.21.202.66:35281)
I20260806 19:42:35.106442 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.103234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:50221 (local address 127.21.202.67:42829)
0806 19:42:35.103675 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:35.103679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:35.104251 (+ 572us) server_negotiation.cc:407] Connection header received
0806 19:42:35.104468 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:35.104476 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:35.104544 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:35.104649 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:35.105766 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:35.105772 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:35.105778 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:35.105796 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:35.105832 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:35.105846 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:35.105976 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:35.106109 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 19:42:35.106170 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305}
I20260806 19:42:35.106423 27558 raft_consensus.cc:482] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:35.106732 27559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.102957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:42829 (local address 127.21.202.65:50221)
0806 19:42:35.104108 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:35.104137 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:35.104286 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:35.104693 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:35.104698 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:35.104722 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:35.105564 (+ 842us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:42:35.105566 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:35.105582 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:42:35.105620 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:35.105949 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:35.105954 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:35.105959 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 19:42:35.106110 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 19:42:35.106171 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"thread_start_us":513,"threads_started":1}
I20260806 19:42:35.106765 27545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.103351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:42985 (local address 127.21.202.66:35281)
0806 19:42:35.103572 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:35.103578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:35.104546 (+ 968us) server_negotiation.cc:407] Connection header received
0806 19:42:35.104691 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:35.104696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:35.104758 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:35.104906 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:35.105983 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:35.105990 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:35.105997 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:35.106020 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:35.106064 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:35.106083 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:35.106197 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:35.106417 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 19:42:35.106497 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 19:42:35.107062 27558 raft_consensus.cc:504] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "701da40e5e94457b8b04980436e43b41" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42829 } } peers { permanent_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35281 } } peers { permanent_uuid: "ac176c40c94646168bd577919dac1ac4" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39541 } }
I20260806 19:42:35.107440 27560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.103334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:35281 (local address 127.21.202.65:42985)
0806 19:42:35.104418 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:35.104435 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:35.104559 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:35.104934 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:35.104937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:35.104978 (+ 41us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:35.105764 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:42:35.105767 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:35.105784 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:42:35.105828 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:35.106248 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:35.106253 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:35.106258 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 19:42:35.106977 (+ 719us) client_negotiation.cc:240] Negotiation successful
0806 19:42:35.107033 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"thread_start_us":642,"threads_started":1}
I20260806 19:42:35.107942 27486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86667cc78ed44b07aa357dfd2db18928" candidate_uuid: "ac176c40c94646168bd577919dac1ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "701da40e5e94457b8b04980436e43b41" is_pre_election: true
I20260806 19:42:35.108392 27486 raft_consensus.cc:2454] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac176c40c94646168bd577919dac1ac4 in term 0.
I20260806 19:42:35.108388 27354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86667cc78ed44b07aa357dfd2db18928" candidate_uuid: "ac176c40c94646168bd577919dac1ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" is_pre_election: true
I20260806 19:42:35.108866 27354 raft_consensus.cc:2454] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac176c40c94646168bd577919dac1ac4 in term 0.
I20260806 19:42:35.109067 27155 leader_election.cc:302] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 701da40e5e94457b8b04980436e43b41, ac176c40c94646168bd577919dac1ac4; no voters:
I20260806 19:42:35.109499 27556 raft_consensus.cc:2781] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:42:35.109647 27556 raft_consensus.cc:482] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:35.109747 27556 raft_consensus.cc:3037] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:35.112334 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.109680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:57505 (local address 127.21.202.67:42829)
0806 19:42:35.109852 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:35.109858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:35.110045 (+ 187us) server_negotiation.cc:407] Connection header received
0806 19:42:35.110167 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:35.110172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:35.110229 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:35.110334 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:35.111675 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:35.111681 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:35.111686 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:35.111704 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:35.111739 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:35.111753 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:35.111896 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:35.112015 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 19:42:35.112068 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 19:42:35.112763 27561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.109327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:42829 (local address 127.21.202.66:57505)
0806 19:42:35.109939 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:35.109957 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:35.110069 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:35.110381 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:35.110387 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:35.110416 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:35.111420 (+ 1004us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:42:35.111426 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:35.111450 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:42:35.111512 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:35.111879 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:35.111886 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:35.111902 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 19:42:35.112075 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 19:42:35.112143 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"spinlock_wait_cycles":18816,"thread_start_us":406,"threads_started":1}
I20260806 19:42:35.112962 27558 leader_election.cc:288] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 701da40e5e94457b8b04980436e43b41 (127.21.202.67:42829), ac176c40c94646168bd577919dac1ac4 (127.21.202.65:39541)
I20260806 19:42:35.112883 27556 raft_consensus.cc:504] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "701da40e5e94457b8b04980436e43b41" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42829 } } peers { permanent_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35281 } } peers { permanent_uuid: "ac176c40c94646168bd577919dac1ac4" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39541 } }
I20260806 19:42:35.113358 27486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86667cc78ed44b07aa357dfd2db18928" candidate_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "701da40e5e94457b8b04980436e43b41" is_pre_election: true
I20260806 19:42:35.113603 27486 raft_consensus.cc:2454] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 287da06b2e6a44b3a1be9eb4a68f8319 in term 0.
I20260806 19:42:35.114087 27556 leader_election.cc:288] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [CANDIDATE]: Term 1 election: Requested vote from peers 701da40e5e94457b8b04980436e43b41 (127.21.202.67:42829), 287da06b2e6a44b3a1be9eb4a68f8319 (127.21.202.66:35281)
I20260806 19:42:35.114043 27354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86667cc78ed44b07aa357dfd2db18928" candidate_uuid: "ac176c40c94646168bd577919dac1ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "287da06b2e6a44b3a1be9eb4a68f8319"
I20260806 19:42:35.114107 27486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86667cc78ed44b07aa357dfd2db18928" candidate_uuid: "ac176c40c94646168bd577919dac1ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "701da40e5e94457b8b04980436e43b41"
I20260806 19:42:35.114266 27354 raft_consensus.cc:3037] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:35.114287 27486 raft_consensus.cc:3037] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:35.114261 27289 leader_election.cc:302] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 287da06b2e6a44b3a1be9eb4a68f8319, 701da40e5e94457b8b04980436e43b41; no voters:
I20260806 19:42:35.117636 27486 raft_consensus.cc:2454] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac176c40c94646168bd577919dac1ac4 in term 1.
I20260806 19:42:35.117712 27354 raft_consensus.cc:2454] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac176c40c94646168bd577919dac1ac4 in term 1.
I20260806 19:42:35.117884 27558 raft_consensus.cc:2741] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 19:42:35.118069 27155 leader_election.cc:302] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 701da40e5e94457b8b04980436e43b41, ac176c40c94646168bd577919dac1ac4; no voters:
I20260806 19:42:35.118572 27556 raft_consensus.cc:2781] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:35.118882 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.110651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:41337 (local address 127.21.202.65:39541)
0806 19:42:35.110815 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:35.110820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:35.113100 (+ 2280us) server_negotiation.cc:407] Connection header received
0806 19:42:35.113363 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:35.113370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:35.113430 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:35.113498 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:35.117747 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:35.117753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:35.117758 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:35.117785 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:35.117828 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:35.117843 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:35.117962 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:35.118490 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 19:42:35.118561 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 19:42:35.119089 27561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.110278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:39541 (local address 127.21.202.66:41337)
0806 19:42:35.113017 (+ 2739us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:35.113035 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:35.113195 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:35.116910 (+ 3715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:35.116916 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:35.116946 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:35.117544 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:42:35.117546 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:35.117560 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:42:35.117596 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:35.118228 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:35.118235 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:35.118240 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 19:42:35.118471 (+ 231us) client_negotiation.cc:240] Negotiation successful
0806 19:42:35.118530 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2642,"thread_start_us":3468,"threads_started":1}
I20260806 19:42:35.119434 27556 raft_consensus.cc:686] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [term 1 LEADER]: Becoming Leader. State: Replica: ac176c40c94646168bd577919dac1ac4, State: Running, Role: LEADER
I20260806 19:42:35.119948 27220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86667cc78ed44b07aa357dfd2db18928" candidate_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac176c40c94646168bd577919dac1ac4" is_pre_election: true
I20260806 19:42:35.119994 27556 consensus_queue.cc:237] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "701da40e5e94457b8b04980436e43b41" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42829 } } peers { permanent_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35281 } } peers { permanent_uuid: "ac176c40c94646168bd577919dac1ac4" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39541 } }
I20260806 19:42:35.124634 27078 catalog_manager.cc:5495] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac176c40c94646168bd577919dac1ac4 (127.21.202.65). New cstate: current_term: 1 leader_uuid: "ac176c40c94646168bd577919dac1ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "701da40e5e94457b8b04980436e43b41" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac176c40c94646168bd577919dac1ac4" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39541 } health_report { overall_health: HEALTHY } } }
I20260806 19:42:35.151880 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.149383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43524 (local address 127.21.202.65:39541)
0806 19:42:35.149585 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:35.149591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:35.149613 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:42:35.149816 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:35.149822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:35.149941 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:35.150062 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:35.151190 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:35.151197 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:35.151202 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:35.151221 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:35.151259 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:35.151273 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:35.151405 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:35.151516 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 19:42:35.151576 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 19:42:35.160401 27354 raft_consensus.cc:1261] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [term 1 FOLLOWER]: Refusing update from remote peer ac176c40c94646168bd577919dac1ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:35.160517 27486 raft_consensus.cc:1261] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41 [term 1 FOLLOWER]: Refusing update from remote peer ac176c40c94646168bd577919dac1ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:35.161458 27563 consensus_queue.cc:1035] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "287da06b2e6a44b3a1be9eb4a68f8319" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:35.161828 27556 consensus_queue.cc:1035] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "701da40e5e94457b8b04980436e43b41" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:35.173276 27566 mvcc.cc:205] Tried to move back new op lower bound from 7315641774727172096 to 7315641774575661056. Current Snapshot: MvccSnapshot[applied={T|T < 7315641774727172096}]
I20260806 19:42:35.188609 27077 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:35.188934 27077 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:35.192607 27077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca3f16e5f57a4611a74acea8a5e22d74: Sending DeleteTablet for 3 replicas of tablet 86667cc78ed44b07aa357dfd2db18928
I20260806 19:42:35.193764 27466 tablet_service.cc:1514] Processing DeleteTablet for tablet 86667cc78ed44b07aa357dfd2db18928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:35 UTC) from {username='slave'} at 127.0.0.1:41418
I20260806 19:42:35.193770 27333 tablet_service.cc:1514] Processing DeleteTablet for tablet 86667cc78ed44b07aa357dfd2db18928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:35 UTC) from {username='slave'} at 127.0.0.1:38370
I20260806 19:42:35.196589 27200 tablet_service.cc:1514] Processing DeleteTablet for tablet 86667cc78ed44b07aa357dfd2db18928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:35 UTC) from {username='slave'} at 127.0.0.1:43518
I20260806 19:42:35.205866 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 27137
I20260806 19:42:35.205883 27577 tablet_replica.cc:330] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4: stopping tablet replica
I20260806 19:42:35.206390 27577 raft_consensus.cc:2229] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:42:35.207167 27577 raft_consensus.cc:2258] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:35.209609 27577 ts_tablet_manager.cc:1905] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:35.217379 27575 tablet_replica.cc:330] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319: stopping tablet replica
I20260806 19:42:35.217885 27575 raft_consensus.cc:2229] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:35.218473 27575 raft_consensus.cc:2258] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:35.219235 27576 tablet_replica.cc:330] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41: stopping tablet replica
I20260806 19:42:35.219679 27576 raft_consensus.cc:2229] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:35.220166 27576 raft_consensus.cc:2258] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:35.222558 27575 ts_tablet_manager.cc:1905] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:35.223711 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.209553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43538 (local address 127.21.202.65:39541)
0806 19:42:35.209758 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:35.209767 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:35.220965 (+ 11198us) server_negotiation.cc:407] Connection header received
0806 19:42:35.221405 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:35.221410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:35.221658 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:35.221825 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:35.223014 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:35.223025 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:35.223029 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:35.223047 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:35.223077 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:35.223089 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:35.223236 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:35.223326 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 19:42:35.223399 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 19:42:35.224447 27261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:35.226337 27576 ts_tablet_manager.cc:1905] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:35.230064 27575 ts_tablet_manager.cc:1918] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:35.230242 27575 log.cc:1196] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/86667cc78ed44b07aa357dfd2db18928
I20260806 19:42:35.230793 27575 ts_tablet_manager.cc:1939] T 86667cc78ed44b07aa357dfd2db18928 P 287da06b2e6a44b3a1be9eb4a68f8319: Deleting consensus metadata
I20260806 19:42:35.232739 27064 catalog_manager.cc:4841] TS 287da06b2e6a44b3a1be9eb4a68f8319 (127.21.202.66:35281): tablet 86667cc78ed44b07aa357dfd2db18928 (table test-table [id=7961c9e265254779970ede9bdccb6a65]) successfully deleted
I20260806 19:42:35.233327 27576 ts_tablet_manager.cc:1918] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:35.233525 27576 log.cc:1196] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/86667cc78ed44b07aa357dfd2db18928
I20260806 19:42:35.234243 27576 ts_tablet_manager.cc:1939] T 86667cc78ed44b07aa357dfd2db18928 P 701da40e5e94457b8b04980436e43b41: Deleting consensus metadata
I20260806 19:42:35.236061 27065 catalog_manager.cc:4841] TS 701da40e5e94457b8b04980436e43b41 (127.21.202.67:42829): tablet 86667cc78ed44b07aa357dfd2db18928 (table test-table [id=7961c9e265254779970ede9bdccb6a65]) successfully deleted
I20260806 19:42:35.325516 27577 ts_tablet_manager.cc:1918] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:35.325812 27577 log.cc:1196] T 86667cc78ed44b07aa357dfd2db18928 P ac176c40c94646168bd577919dac1ac4: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/86667cc78ed44b07aa357dfd2db18928
I20260806 19:42:35.325937 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 27137
I20260806 19:42:35.345412 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 27271
W20260806 19:42:35.345455 27063 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.202.65:39541 (error 108)
I20260806 19:42:35.348289 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.346967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:39541 (local address 127.0.0.1:43548)
0806 19:42:35.347842 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:35.348051 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":631,"threads_started":1}
W20260806 19:42:35.348526 27063 catalog_manager.cc:4584] TS ac176c40c94646168bd577919dac1ac4 (127.21.202.65:39541): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86667cc78ed44b07aa357dfd2db18928: Network error: Client connection negotiation failed: client connection to 127.21.202.65:39541: connect: Connection refused (error 111)
I20260806 19:42:35.349038 27545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.346272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38386 (local address 127.21.202.66:35281)
0806 19:42:35.346477 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:35.346484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:35.346633 (+ 149us) server_negotiation.cc:407] Connection header received
0806 19:42:35.346852 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:35.346857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:35.346986 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:35.347111 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:35.348125 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:35.348131 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:35.348135 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:35.348153 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:35.348192 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:35.348207 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:35.348360 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:35.348547 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 19:42:35.348638 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 19:42:35.349711 27395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:35.402482 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.401849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:39541 (local address 127.0.0.1:43554)
0806 19:42:35.402158 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:35.402254 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260806 19:42:35.403556 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.403183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:39541 (local address 127.0.0.1:43568)
0806 19:42:35.403343 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:35.403429 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 19:42:35.447234 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 27271
I20260806 19:42:35.464375 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.463896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:39541 (local address 127.0.0.1:43572)
0806 19:42:35.464129 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:35.464221 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 19:42:35.465524 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.465126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:39541 (local address 127.0.0.1:43588)
0806 19:42:35.465311 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:35.465382 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 19:42:35.466444 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 27403
I20260806 19:42:35.469296 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.467196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41430 (local address 127.21.202.67:42829)
0806 19:42:35.467384 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:35.467389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:35.467415 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:42:35.467566 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:35.467570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:35.467648 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:35.467735 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:35.468657 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:35.468662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:35.468665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:35.468682 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:35.468716 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:35.468728 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:35.468909 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:35.469039 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 19:42:35.469089 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 19:42:35.470005 27526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:35.546300 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.545776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:39541 (local address 127.0.0.1:43592)
0806 19:42:35.546044 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:35.546138 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260806 19:42:35.547570 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:35.547106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:39541 (local address 127.0.0.1:43598)
0806 19:42:35.547343 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:35.547426 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260806 19:42:35.560686 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 27403
I20260806 19:42:35.579000 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 27046
I20260806 19:42:35.580248 27108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:35.689289 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 27046
2026-08-06T19:42:35Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3045 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 19:42:35.738796 22313 mini_oidc.cc:152] Starting JWKS server
I20260806 19:42:35.743438 22313 webserver.cc:460] Webserver started at https://127.0.0.1:37735/ using document root <none> and password file <none>
I20260806 19:42:35.743791 22313 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 19:42:35.744570 22313 webserver.cc:460] Webserver started at http://127.0.0.1:39475/ using document root <none> and password file <none>
2026-08-06T19:42:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:35Z Disabled control of system clock
I20260806 19:42:35.782979 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:37099
--webserver_interface=127.21.202.126
--webserver_port=0
--builtin_ntp_servers=127.21.202.84:36655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:37099
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37735/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/testchainca.cert with env {}
W20260806 19:42:36.125233 27590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:36.125541 27590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:36.125717 27590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:36.133081 27590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:36.133212 27590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:36.133306 27590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:36.133424 27590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:36.133538 27590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 19:42:36.133658 27590 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 19:42:36.133749 27590 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37735/jwks.json
I20260806 19:42:36.144037 27590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:36655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:37099
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:37099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37735/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:36.145686 27590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:36.147081 27590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:36.159758 27595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:36.159976 27598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:36.159701 27596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:36.161005 27590 server_base.cc:1034] running on GCE node
I20260806 19:42:36.161775 27590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:36.163297 27590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:36.164534 27590 hybrid_clock.cc:648] HybridClock initialized: now 1786045356164496 us; error 43 us; skew 500 ppm
I20260806 19:42:36.164986 27590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:42:36.167434 27590 webserver.cc:460] Webserver started at http://127.21.202.126:36981/ using document root <none> and password file <none>
I20260806 19:42:36.168156 27590 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:36.168256 27590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:36.168637 27590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:36.171705 27590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "e2c905a85af94618afcc1bd9f5ed41f0"
format_stamp: "Formatted at 2026-08-06 19:42:36 on dist-test-slave-77v9"
server_key: "2bcacabd5a849f0e5cf2ff4116e7fcfc"
server_key_iv: "9182c6ce0078e937e3678fa0038c4287"
server_key_version: "encryptionkey@0"
I20260806 19:42:36.172556 27590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "e2c905a85af94618afcc1bd9f5ed41f0"
format_stamp: "Formatted at 2026-08-06 19:42:36 on dist-test-slave-77v9"
server_key: "2bcacabd5a849f0e5cf2ff4116e7fcfc"
server_key_iv: "9182c6ce0078e937e3678fa0038c4287"
server_key_version: "encryptionkey@0"
I20260806 19:42:36.178140 27590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 19:42:36.181775 27604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:36.183319 27590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:42:36.183552 27590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "e2c905a85af94618afcc1bd9f5ed41f0"
format_stamp: "Formatted at 2026-08-06 19:42:36 on dist-test-slave-77v9"
server_key: "2bcacabd5a849f0e5cf2ff4116e7fcfc"
server_key_iv: "9182c6ce0078e937e3678fa0038c4287"
server_key_version: "encryptionkey@0"
I20260806 19:42:36.183810 27590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:36.200788 27590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:36.201835 27590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:36.202224 27590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:36.223148 27590 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:37099
I20260806 19:42:36.223166 27655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:37099 every 8 connection(s)
I20260806 19:42:36.224753 27590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:36.227120 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 27590
I20260806 19:42:36.227326 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:36.228161 22313 external_mini_cluster.cc:1428] Setting key 01e0e09770aeb52476d8d56b3ccdd6d6
I20260806 19:42:36.229785 27656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:36.242570 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:36.230208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50532 (local address 127.21.202.126:37099)
0806 19:42:36.231196 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:36.231207 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:36.231251 (+ 44us) server_negotiation.cc:407] Connection header received
0806 19:42:36.232873 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:36.232903 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:36.233365 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:36.233711 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:36.234445 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:36.235755 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:36.237701 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:36.238328 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:36.238469 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:36.240030 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:36.240038 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:36.240051 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:36.240641 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:36.240717 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:36.240737 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:36.240917 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:36.241151 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 19:42:36.241586 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":389,"threads_started":1}
I20260806 19:42:36.244035 27656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0: Bootstrap starting.
I20260806 19:42:36.246762 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:42:36.247632 27656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:36.248613 27656 log.cc:824] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:36.251619 27656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0: No bootstrap required, opened a new log
I20260806 19:42:36.260000 27656 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c905a85af94618afcc1bd9f5ed41f0" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37099 } }
I20260806 19:42:36.260469 27656 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:36.260596 27656 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c905a85af94618afcc1bd9f5ed41f0, State: Initialized, Role: FOLLOWER
I20260806 19:42:36.261314 27656 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c905a85af94618afcc1bd9f5ed41f0" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37099 } }
I20260806 19:42:36.261600 27656 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:36.261777 27656 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:36.262012 27656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:36.264421 27656 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c905a85af94618afcc1bd9f5ed41f0" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37099 } }
I20260806 19:42:36.265022 27656 leader_election.cc:302] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2c905a85af94618afcc1bd9f5ed41f0; no voters:
I20260806 19:42:36.265646 27656 leader_election.cc:288] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:36.265861 27661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:36.267230 27661 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [term 1 LEADER]: Becoming Leader. State: Replica: e2c905a85af94618afcc1bd9f5ed41f0, State: Running, Role: LEADER
I20260806 19:42:36.267743 27661 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c905a85af94618afcc1bd9f5ed41f0" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37099 } }
I20260806 19:42:36.268226 27656 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:36.270795 27662 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2c905a85af94618afcc1bd9f5ed41f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c905a85af94618afcc1bd9f5ed41f0" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37099 } } }
I20260806 19:42:36.271314 27662 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:36.270833 27663 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2c905a85af94618afcc1bd9f5ed41f0. Latest consensus state: current_term: 1 leader_uuid: "e2c905a85af94618afcc1bd9f5ed41f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c905a85af94618afcc1bd9f5ed41f0" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37099 } } }
I20260806 19:42:36.271816 27663 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:36.277875 27669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:36.283136 27669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:36.293339 27669 catalog_manager.cc:1347] Generated new cluster ID: e3faa86fe75748c3b814a43f81472e2b
I20260806 19:42:36.293473 27669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:36.314383 27669 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:36.316241 27669 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:36.330046 27669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2c905a85af94618afcc1bd9f5ed41f0: Generated new TSK 0
I20260806 19:42:36.331146 27669 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 19:42:36.388504 27686 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 19:42:36.388788 27686 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 19:42:36.391194 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:36.358913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50540 (local address 127.21.202.126:37099)
0806 19:42:36.359173 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:36.359184 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:36.359752 (+ 568us) server_negotiation.cc:407] Connection header received
0806 19:42:36.360212 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:36.360220 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:36.360305 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:36.360437 (+ 132us) server_negotiation.cc:226] Negotiated authn=JWT
0806 19:42:36.361201 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:36.362415 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:36.364534 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:36.365054 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:36.365151 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:36.365790 (+ 639us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 19:42:36.390433 (+ 24643us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37735/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 19:42:36.390853 (+ 420us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50540: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37735/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":75}
W20260806 19:42:36.391330 27659 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50540: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37735/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 19:42:36.393136 27685 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:42:36.358915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:37099 (local address 127.0.0.1:50540)
0806 19:42:36.359635 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:36.359657 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:36.359856 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:36.360501 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:36.360507 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:36.360522 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0806 19:42:36.360987 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:36.361001 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:36.362560 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:36.362566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:36.364392 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:36.364407 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:36.365235 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:36.365242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:36.365433 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:36.365510 (+ 77us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 19:42:36.390790 (+ 25280us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37735/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 19:42:36.391111 (+ 321us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.202.126:37099: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37735/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":376,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.21.202.126:37099: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37735/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 19:42:36.394661 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 27590
I20260806 19:42:36.396015 27651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:36.501160 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 27590
2026-08-06T19:42:36Z chronyd exiting
I20260806 19:42:36.554087 22313 test_util.cc:175] -----------------------------------------------
I20260806 19:42:36.554206 22313 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786045310306758-22313-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (824 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 19:42:36.557930 22313 mini_oidc.cc:152] Starting JWKS server
I20260806 19:42:36.559726 22313 webserver.cc:460] Webserver started at https://127.0.0.1:39609/ using document root <none> and password file <none>
I20260806 19:42:36.559949 22313 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 19:42:36.560571 22313 webserver.cc:460] Webserver started at http://127.0.0.1:36665/ using document root <none> and password file <none>
2026-08-06T19:42:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:36Z Disabled control of system clock
I20260806 19:42:36.594843 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:45997
--webserver_interface=127.21.202.126
--webserver_port=0
--builtin_ntp_servers=127.21.202.84:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:45997
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39609/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/testchainca.cert with env {}
W20260806 19:42:36.958243 27694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:36.958593 27694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:36.958750 27694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:36.966044 27694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:36.966198 27694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:36.966291 27694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:36.966406 27694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:36.966518 27694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 19:42:36.966609 27694 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 19:42:36.966686 27694 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39609/jwks.json
I20260806 19:42:36.976647 27694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:45997
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:45997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39609/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:36.978150 27694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:36.979472 27694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:36.991448 27700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:36.991410 27701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:36.991478 27703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:36.992523 27694 server_base.cc:1034] running on GCE node
I20260806 19:42:36.993346 27694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:36.994768 27694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:36.996002 27694 hybrid_clock.cc:648] HybridClock initialized: now 1786045356995966 us; error 33 us; skew 500 ppm
I20260806 19:42:36.996351 27694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:42:36.999003 27694 webserver.cc:460] Webserver started at http://127.21.202.126:35879/ using document root <none> and password file <none>
I20260806 19:42:36.999759 27694 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:36.999912 27694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:37.000293 27694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:37.003543 27694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "1e01e6995c7543a6b81902cd39207843"
format_stamp: "Formatted at 2026-08-06 19:42:36 on dist-test-slave-77v9"
server_key: "f969de6515d82d51f4248b60bdf4c25d"
server_key_iv: "cd6973dba8b7ba72825277868aa70666"
server_key_version: "encryptionkey@0"
I20260806 19:42:37.004477 27694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "1e01e6995c7543a6b81902cd39207843"
format_stamp: "Formatted at 2026-08-06 19:42:36 on dist-test-slave-77v9"
server_key: "f969de6515d82d51f4248b60bdf4c25d"
server_key_iv: "cd6973dba8b7ba72825277868aa70666"
server_key_version: "encryptionkey@0"
I20260806 19:42:37.010383 27694 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 19:42:37.014772 27710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:37.016273 27694 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 19:42:37.016496 27694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "1e01e6995c7543a6b81902cd39207843"
format_stamp: "Formatted at 2026-08-06 19:42:36 on dist-test-slave-77v9"
server_key: "f969de6515d82d51f4248b60bdf4c25d"
server_key_iv: "cd6973dba8b7ba72825277868aa70666"
server_key_version: "encryptionkey@0"
I20260806 19:42:37.016757 27694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:37.037966 27694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:37.038842 27694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:37.039211 27694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:37.059098 27694 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:45997
I20260806 19:42:37.059136 27762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:45997 every 8 connection(s)
I20260806 19:42:37.060636 27694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:37.064649 27763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:37.070195 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 27694
I20260806 19:42:37.070437 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:37.071105 22313 external_mini_cluster.cc:1428] Setting key d343f44f3ff2077bde0ea14a97dee877
I20260806 19:42:37.075634 27763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843: Bootstrap starting.
I20260806 19:42:37.079101 27763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:37.080377 27763 log.cc:824] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:37.084190 27763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843: No bootstrap required, opened a new log
I20260806 19:42:37.085862 27766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:37.073331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54918 (local address 127.21.202.126:45997)
0806 19:42:37.074384 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:37.074395 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:37.074439 (+ 44us) server_negotiation.cc:407] Connection header received
0806 19:42:37.076160 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:37.076183 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:37.076620 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:37.077027 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:37.077634 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:37.078895 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:37.080780 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:37.081975 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:37.082135 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:37.083491 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:37.083500 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:37.083515 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:37.084088 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:37.084172 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:37.084197 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:37.084285 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:37.084589 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 19:42:37.084960 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":397,"threads_started":1}
I20260806 19:42:37.090242 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:42:37.095626 27763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e01e6995c7543a6b81902cd39207843" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45997 } }
I20260806 19:42:37.096004 27763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:37.096086 27763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e01e6995c7543a6b81902cd39207843, State: Initialized, Role: FOLLOWER
I20260806 19:42:37.096634 27763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e01e6995c7543a6b81902cd39207843" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45997 } }
I20260806 19:42:37.096926 27763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:37.097051 27763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:37.097191 27763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:37.099565 27763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e01e6995c7543a6b81902cd39207843" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45997 } }
I20260806 19:42:37.100054 27763 leader_election.cc:302] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e01e6995c7543a6b81902cd39207843; no voters:
I20260806 19:42:37.100739 27763 leader_election.cc:288] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:37.100987 27768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:37.102288 27768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [term 1 LEADER]: Becoming Leader. State: Replica: 1e01e6995c7543a6b81902cd39207843, State: Running, Role: LEADER
I20260806 19:42:37.102813 27768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e01e6995c7543a6b81902cd39207843" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45997 } }
I20260806 19:42:37.103242 27763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:37.105618 27769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e01e6995c7543a6b81902cd39207843" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e01e6995c7543a6b81902cd39207843" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45997 } } }
I20260806 19:42:37.106069 27769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:37.106004 27770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e01e6995c7543a6b81902cd39207843. Latest consensus state: current_term: 1 leader_uuid: "1e01e6995c7543a6b81902cd39207843" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e01e6995c7543a6b81902cd39207843" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45997 } } }
I20260806 19:42:37.106304 27770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:37.107949 27774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:37.113525 27774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:37.123103 27774 catalog_manager.cc:1347] Generated new cluster ID: ba06a3dd166b470eb190267c7a72e166
I20260806 19:42:37.123255 27774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:37.132697 27774 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:37.134231 27774 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:37.144244 27774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e01e6995c7543a6b81902cd39207843: Generated new TSK 0
I20260806 19:42:37.145181 27774 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 19:42:37.177521 27794 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 19:42:37.177687 27794 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 19:42:37.178952 27766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:37.158315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54922 (local address 127.21.202.126:45997)
0806 19:42:37.158470 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:37.158475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:37.158878 (+ 403us) server_negotiation.cc:407] Connection header received
0806 19:42:37.159131 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:37.159137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:37.159235 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:37.159411 (+ 176us) server_negotiation.cc:226] Negotiated authn=JWT
0806 19:42:37.159839 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:37.161009 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:37.162480 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:37.163150 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:37.163274 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:37.163736 (+ 462us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 19:42:37.178230 (+ 14494us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 19:42:37.178644 (+ 414us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54922: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 19:42:37.179086 27766 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54922: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 19:42:37.179153 27792 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:42:37.158155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45997 (local address 127.0.0.1:54922)
0806 19:42:37.158775 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:37.158792 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:37.158920 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:37.159424 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:37.159428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:37.159438 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0806 19:42:37.159668 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:37.159675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:37.161226 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:37.161231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:37.162329 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:37.162341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:37.163311 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:37.163315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:37.163445 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:37.163476 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 19:42:37.178509 (+ 15033us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 19:42:37.178744 (+ 235us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.202.126:45997: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":338,"threads_started":1}
I20260806 19:42:37.180931 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 27694
I20260806 19:42:37.182199 27758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:37.278105 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 27694
2026-08-06T19:42:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (777 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 19:42:37.335390 22313 mini_oidc.cc:152] Starting JWKS server
I20260806 19:42:37.337401 22313 webserver.cc:460] Webserver started at https://127.0.0.1:37689/ using document root <none> and password file <none>
I20260806 19:42:37.337670 22313 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 19:42:37.338354 22313 webserver.cc:460] Webserver started at http://127.0.0.1:33523/ using document root <none> and password file <none>
2026-08-06T19:42:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:37Z Disabled control of system clock
I20260806 19:42:37.373636 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:38069
--webserver_interface=127.21.202.126
--webserver_port=0
--builtin_ntp_servers=127.21.202.84:42071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:38069
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37689/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 19:42:37.712927 27801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:37.713306 27801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:37.713467 27801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:37.720891 27801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:37.721046 27801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:37.721134 27801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:37.721241 27801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:37.721318 27801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 19:42:37.721408 27801 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 19:42:37.721488 27801 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37689/jwks.json
I20260806 19:42:37.731540 27801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:38069
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:38069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37689/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:37.733034 27801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:37.734959 27801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:37.748418 27806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:37.748368 27807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:37.748390 27809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:37.749725 27801 server_base.cc:1034] running on GCE node
I20260806 19:42:37.750559 27801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:37.752018 27801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:37.753263 27801 hybrid_clock.cc:648] HybridClock initialized: now 1786045357753213 us; error 63 us; skew 500 ppm
I20260806 19:42:37.753744 27801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:42:37.756412 27801 webserver.cc:460] Webserver started at http://127.21.202.126:36387/ using document root <none> and password file <none>
I20260806 19:42:37.757289 27801 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:37.757443 27801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:37.757812 27801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:37.760959 27801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "3186909293fe435db92f3c64eda7b1df"
format_stamp: "Formatted at 2026-08-06 19:42:37 on dist-test-slave-77v9"
server_key: "77479edd573bcc5a09741f8e5e7c36d6"
server_key_iv: "cdc4e7ae65e42c402fe39cf9f21aea39"
server_key_version: "encryptionkey@0"
I20260806 19:42:37.761863 27801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "3186909293fe435db92f3c64eda7b1df"
format_stamp: "Formatted at 2026-08-06 19:42:37 on dist-test-slave-77v9"
server_key: "77479edd573bcc5a09741f8e5e7c36d6"
server_key_iv: "cdc4e7ae65e42c402fe39cf9f21aea39"
server_key_version: "encryptionkey@0"
I20260806 19:42:37.767516 27801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:42:37.771166 27815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:37.772516 27801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 19:42:37.772715 27801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "3186909293fe435db92f3c64eda7b1df"
format_stamp: "Formatted at 2026-08-06 19:42:37 on dist-test-slave-77v9"
server_key: "77479edd573bcc5a09741f8e5e7c36d6"
server_key_iv: "cdc4e7ae65e42c402fe39cf9f21aea39"
server_key_version: "encryptionkey@0"
I20260806 19:42:37.773000 27801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:37.788211 27801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:37.789119 27801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:37.789462 27801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:37.808969 27801 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:38069
I20260806 19:42:37.809008 27867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:38069 every 8 connection(s)
I20260806 19:42:37.810384 27801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:37.814160 27868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:37.816397 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 27801
I20260806 19:42:37.816617 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:37.817348 22313 external_mini_cluster.cc:1428] Setting key 5d6db4f77d11e670235e35a474561cfc
I20260806 19:42:37.827145 27868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df: Bootstrap starting.
I20260806 19:42:37.831235 27868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:37.832491 27868 log.cc:824] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:37.832561 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:37.819619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59442 (local address 127.21.202.126:38069)
0806 19:42:37.820696 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:37.820709 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:37.820765 (+ 56us) server_negotiation.cc:407] Connection header received
0806 19:42:37.822668 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:37.822693 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:37.823222 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:37.823604 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:37.824332 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:37.825739 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:37.827810 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:37.828512 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:37.828648 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:37.830133 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:37.830141 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:37.830154 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:37.830705 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:37.830844 (+ 139us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:37.830866 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:37.830964 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:37.831246 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 19:42:37.831574 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":402,"threads_started":1}
I20260806 19:42:37.836181 27868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df: No bootstrap required, opened a new log
I20260806 19:42:37.837309 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:42:37.845759 27868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3186909293fe435db92f3c64eda7b1df" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38069 } }
I20260806 19:42:37.846175 27868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:37.846320 27868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3186909293fe435db92f3c64eda7b1df, State: Initialized, Role: FOLLOWER
I20260806 19:42:37.846952 27868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3186909293fe435db92f3c64eda7b1df" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38069 } }
I20260806 19:42:37.847198 27868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:37.847366 27868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:37.847553 27868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:37.850366 27868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3186909293fe435db92f3c64eda7b1df" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38069 } }
I20260806 19:42:37.851063 27868 leader_election.cc:302] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3186909293fe435db92f3c64eda7b1df; no voters:
I20260806 19:42:37.851794 27868 leader_election.cc:288] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:37.851995 27874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:37.852778 27874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [term 1 LEADER]: Becoming Leader. State: Replica: 3186909293fe435db92f3c64eda7b1df, State: Running, Role: LEADER
I20260806 19:42:37.853507 27874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3186909293fe435db92f3c64eda7b1df" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38069 } }
I20260806 19:42:37.854060 27868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:37.856505 27876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3186909293fe435db92f3c64eda7b1df. Latest consensus state: current_term: 1 leader_uuid: "3186909293fe435db92f3c64eda7b1df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3186909293fe435db92f3c64eda7b1df" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38069 } } }
I20260806 19:42:37.856963 27876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:37.858073 27875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3186909293fe435db92f3c64eda7b1df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3186909293fe435db92f3c64eda7b1df" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38069 } } }
I20260806 19:42:37.858598 27875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:37.860184 27881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:37.866130 27881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:37.875221 27881 catalog_manager.cc:1347] Generated new cluster ID: c2996c305860446583ef884dae36d8c0
I20260806 19:42:37.875346 27881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:37.892927 27881 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:37.894191 27881 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:37.901229 27881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3186909293fe435db92f3c64eda7b1df: Generated new TSK 0
I20260806 19:42:37.902328 27881 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:42:37.934705 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:37.925166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59454 (local address 127.21.202.126:38069)
0806 19:42:37.925394 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:37.925402 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:37.925866 (+ 464us) server_negotiation.cc:407] Connection header received
0806 19:42:37.926135 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:37.926141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:37.926241 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:37.926444 (+ 203us) server_negotiation.cc:226] Negotiated authn=JWT
0806 19:42:37.927091 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:37.928273 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:37.929921 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:37.930644 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:37.930780 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:37.931474 (+ 694us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 19:42:37.933997 (+ 2523us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37689/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 19:42:37.934352 (+ 355us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59454: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37689/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":87}
W20260806 19:42:37.934852 27872 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59454: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37689/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 19:42:37.936601 27898 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:42:37.925018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:38069 (local address 127.0.0.1:59454)
0806 19:42:37.925730 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:37.925754 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:37.925901 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:37.926508 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:37.926512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:37.926525 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0806 19:42:37.926871 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:37.926881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:37.928450 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:37.928455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:37.929762 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:37.929777 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:37.930929 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:37.930935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:37.931089 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:37.931130 (+ 41us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 19:42:37.934278 (+ 3148us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37689/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 19:42:37.936162 (+ 1884us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.202.126:38069: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37689/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":1531,"thread_start_us":378,"threads_started":1}
I20260806 19:42:37.938292 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 27801
I20260806 19:42:37.939865 27863 generic_service.cc:196] Checking for leaks (request via RPC)
E20260806 19:42:37.948254 27899 webserver.cc:533] Webserver: sslize failed:
E20260806 19:42:37.948359 27899 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 19:42:38.035743 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 27801
2026-08-06T19:42:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (756 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T19:42:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:38Z Disabled control of system clock
I20260806 19:42:38.126907 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:39565
--webserver_interface=127.21.202.126
--webserver_port=0
--builtin_ntp_servers=127.21.202.84:43257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:39565
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/testchainca.cert with env {}
W20260806 19:42:38.462743 27904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:38.463063 27904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:38.463212 27904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:38.470671 27904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:38.470788 27904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:38.470825 27904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:38.470928 27904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:38.470999 27904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 19:42:38.471088 27904 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 19:42:38.471145 27904 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 19:42:38.481789 27904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:43257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:39565
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:39565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:38.483186 27904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:38.484616 27904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:38.496774 27910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:38.496881 27909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:38.497450 27912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:38.498168 27904 server_base.cc:1034] running on GCE node
I20260806 19:42:38.499043 27904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:38.500521 27904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:38.501776 27904 hybrid_clock.cc:648] HybridClock initialized: now 1786045358501681 us; error 112 us; skew 500 ppm
I20260806 19:42:38.502156 27904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:42:38.507732 27904 webserver.cc:460] Webserver started at http://127.21.202.126:33295/ using document root <none> and password file <none>
I20260806 19:42:38.508594 27904 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:38.508755 27904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:38.509135 27904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:38.512606 27904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "24b43fedb29e458c8f318c8c59d533f5"
format_stamp: "Formatted at 2026-08-06 19:42:38 on dist-test-slave-77v9"
server_key: "c6a915aa1cf983a4378fd36f8208467c"
server_key_iv: "1989087c480a81418e5521b8a1bd70f5"
server_key_version: "encryptionkey@0"
I20260806 19:42:38.513643 27904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "24b43fedb29e458c8f318c8c59d533f5"
format_stamp: "Formatted at 2026-08-06 19:42:38 on dist-test-slave-77v9"
server_key: "c6a915aa1cf983a4378fd36f8208467c"
server_key_iv: "1989087c480a81418e5521b8a1bd70f5"
server_key_version: "encryptionkey@0"
I20260806 19:42:38.519421 27904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:42:38.523017 27918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:38.524457 27904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:42:38.524668 27904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "24b43fedb29e458c8f318c8c59d533f5"
format_stamp: "Formatted at 2026-08-06 19:42:38 on dist-test-slave-77v9"
server_key: "c6a915aa1cf983a4378fd36f8208467c"
server_key_iv: "1989087c480a81418e5521b8a1bd70f5"
server_key_version: "encryptionkey@0"
I20260806 19:42:38.524966 27904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:38.543823 27904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:38.544744 27904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:38.545090 27904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:38.564772 27904 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:39565
I20260806 19:42:38.564791 27969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:39565 every 8 connection(s)
I20260806 19:42:38.566220 27904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:38.570163 27970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:38.570569 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 27904
I20260806 19:42:38.570793 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:38.571462 22313 external_mini_cluster.cc:1428] Setting key ec833f8036d3a98e1da5f945a8226c56
I20260806 19:42:38.582587 27970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5: Bootstrap starting.
I20260806 19:42:38.585270 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:38.573384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.21.202.126:39565)
0806 19:42:38.574353 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:38.574363 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:38.574404 (+ 41us) server_negotiation.cc:407] Connection header received
0806 19:42:38.576381 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:38.576403 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:38.576933 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:38.577328 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:38.577874 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:38.579153 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:38.580439 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:38.581193 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:38.581384 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:38.582632 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:38.582639 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:38.582653 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:42:38.583226 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:38.583305 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:42:38.583325 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:38.583467 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:38.583690 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 19:42:38.584229 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":338,"threads_started":1}
I20260806 19:42:38.586232 27970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:38.587589 27970 log.cc:824] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:38.589509 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:42:38.590750 27970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5: No bootstrap required, opened a new log
I20260806 19:42:38.599082 27970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b43fedb29e458c8f318c8c59d533f5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39565 } }
I20260806 19:42:38.599481 27970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:38.599596 27970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24b43fedb29e458c8f318c8c59d533f5, State: Initialized, Role: FOLLOWER
I20260806 19:42:38.600135 27970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b43fedb29e458c8f318c8c59d533f5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39565 } }
I20260806 19:42:38.600361 27970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:38.600512 27970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:38.600682 27970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:38.602820 27970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b43fedb29e458c8f318c8c59d533f5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39565 } }
I20260806 19:42:38.603325 27970 leader_election.cc:302] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24b43fedb29e458c8f318c8c59d533f5; no voters:
I20260806 19:42:38.603950 27970 leader_election.cc:288] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:38.604123 27975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:38.605082 27975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [term 1 LEADER]: Becoming Leader. State: Replica: 24b43fedb29e458c8f318c8c59d533f5, State: Running, Role: LEADER
I20260806 19:42:38.605610 27975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b43fedb29e458c8f318c8c59d533f5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39565 } }
I20260806 19:42:38.606019 27970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:38.609486 27976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24b43fedb29e458c8f318c8c59d533f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b43fedb29e458c8f318c8c59d533f5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39565 } } }
I20260806 19:42:38.609477 27977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24b43fedb29e458c8f318c8c59d533f5. Latest consensus state: current_term: 1 leader_uuid: "24b43fedb29e458c8f318c8c59d533f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b43fedb29e458c8f318c8c59d533f5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39565 } } }
I20260806 19:42:38.609903 27977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:38.609899 27976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:38.612221 27984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:38.617391 27984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:38.625663 27984 catalog_manager.cc:1347] Generated new cluster ID: c486016b47cd433fa837a1d542e01932
I20260806 19:42:38.625772 27984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:38.641387 27984 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:38.643167 27984 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:38.662569 27984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24b43fedb29e458c8f318c8c59d533f5: Generated new TSK 0
I20260806 19:42:38.663748 27984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:42:38.800688 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:38.682830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.21.202.126:39565)
0806 19:42:38.683060 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:38.683067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:38.683305 (+ 238us) server_negotiation.cc:407] Connection header received
0806 19:42:38.683588 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:38.683594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:38.683688 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:38.683872 (+ 184us) server_negotiation.cc:226] Negotiated authn=JWT
0806 19:42:38.684313 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:38.685527 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:38.686942 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:38.687671 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:38.687798 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:38.688332 (+ 534us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 19:42:38.799898 (+111566us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 19:42:38.800357 (+ 459us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59894: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 19:42:38.800872 27973 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59894: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 19:42:38.800988 27999 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:42:38.682621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:39565 (local address 127.0.0.1:59894)
0806 19:42:38.683184 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:38.683203 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:38.683356 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:38.683884 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:38.683888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:38.683897 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 19:42:38.684163 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:38.684172 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:38.685681 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:38.685685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:38.686801 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:38.686813 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:38.687879 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:38.687884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:38.688015 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:38.688050 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 19:42:38.800210 (+112160us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 19:42:38.800511 (+ 301us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.202.126:39565: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":456,"threads_started":1}
I20260806 19:42:38.802745 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 27904
I20260806 19:42:38.803938 27965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:38.911386 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 27904
2026-08-06T19:42:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (856 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (453 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (458 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:42:39 dist-test-slave-77v9 krb5kdc[28015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:42:39 dist-test-slave-77v9 krb5kdc[28015](info): set up 2 sockets
Aug 06 19:42:39 dist-test-slave-77v9 krb5kdc[28015](info): commencing operation
krb5kdc: starting...
W20260806 19:42:40.974892 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.001s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:42:41 dist-test-slave-77v9 krb5kdc[28015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:42:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:42:41.233281 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42809
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:42809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:41.580482 28028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:41.580907 28028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:41.581066 28028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:41.588629 28028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:41.588795 28028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:41.588958 28028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:41.589061 28028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:41.589143 28028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:42:41.600561 28028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:42809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:41.602144 28028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:41.603603 28028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:41.616155 28034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:41.616288 28033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:41.616262 28036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:41.617093 28028 server_base.cc:1034] running on GCE node
I20260806 19:42:41.617877 28028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:41.619415 28028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:41.620662 28028 hybrid_clock.cc:648] HybridClock initialized: now 1786045361620610 us; error 29 us; skew 500 ppm
I20260806 19:42:41.621136 28028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:41 dist-test-slave-77v9 krb5kdc[28015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045361, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:41.624682 28028 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:42:41.626331 28028 webserver.cc:460] Webserver started at http://127.21.202.126:36247/ using document root <none> and password file <none>
I20260806 19:42:41.629321 28028 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:41.629542 28028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:41.629977 28028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:41.633540 28028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "697dc1d624e64948868a3bcbbfba4833"
format_stamp: "Formatted at 2026-08-06 19:42:41 on dist-test-slave-77v9"
server_key: "8dc280957f11b5b7770a12a4e73238a0"
server_key_iv: "1c9118700f8661f26e9cb1fe5881e1dc"
server_key_version: "encryptionkey@0"
I20260806 19:42:41.634613 28028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "697dc1d624e64948868a3bcbbfba4833"
format_stamp: "Formatted at 2026-08-06 19:42:41 on dist-test-slave-77v9"
server_key: "8dc280957f11b5b7770a12a4e73238a0"
server_key_iv: "1c9118700f8661f26e9cb1fe5881e1dc"
server_key_version: "encryptionkey@0"
I20260806 19:42:41.640602 28028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 19:42:41.644552 28043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:41.646030 28028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:41.646266 28028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "697dc1d624e64948868a3bcbbfba4833"
format_stamp: "Formatted at 2026-08-06 19:42:41 on dist-test-slave-77v9"
server_key: "8dc280957f11b5b7770a12a4e73238a0"
server_key_iv: "1c9118700f8661f26e9cb1fe5881e1dc"
server_key_version: "encryptionkey@0"
I20260806 19:42:41.646540 28028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:41.677452 28028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:41.682893 28028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:41.683239 28028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:41.705610 28028 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:42809
I20260806 19:42:41.705664 28095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:42809 every 8 connection(s)
I20260806 19:42:41.707194 28028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:41.709443 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 28028
I20260806 19:42:41.709743 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:41.710723 22313 external_mini_cluster.cc:1428] Setting key a7e8aabf553b9f9d5d20388ecd18128a
I20260806 19:42:41.712458 28096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 19:42:41 dist-test-slave-77v9 krb5kdc[28015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:41.728713 28096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833: Bootstrap starting.
I20260806 19:42:41.732192 28096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:41.733417 28096 log.cc:824] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:41.736701 28096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833: No bootstrap required, opened a new log
I20260806 19:42:41.737397 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:41.713156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58940 (local address 127.21.202.126:42809)
0806 19:42:41.714068 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:41.714081 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:41.714125 (+ 44us) server_negotiation.cc:407] Connection header received
0806 19:42:41.715883 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:41.715910 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:41.716336 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:41.716705 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:41.718810 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:41.720389 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:41.722445 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:41.723199 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:41.723399 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:41.728772 (+ 5373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:41.728780 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:41.728797 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:41.728913 (+ 116us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:41.732263 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:41.733192 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:41.733199 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:41.733206 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:41.733398 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:41.733854 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:41.733858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:41.733861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:41.734449 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:41.734798 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:41.735285 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 19:42:41.735736 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":417,"threads_started":1}
I20260806 19:42:41.745695 28096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697dc1d624e64948868a3bcbbfba4833" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42809 } }
I20260806 19:42:41.746122 28096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:41.746250 28096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697dc1d624e64948868a3bcbbfba4833, State: Initialized, Role: FOLLOWER
I20260806 19:42:41.746917 28096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697dc1d624e64948868a3bcbbfba4833" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42809 } }
I20260806 19:42:41.747232 28096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:41.747397 28096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:41.747598 28096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:41.750530 28096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697dc1d624e64948868a3bcbbfba4833" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42809 } }
I20260806 19:42:41.751142 28096 leader_election.cc:302] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 697dc1d624e64948868a3bcbbfba4833; no voters:
I20260806 19:42:41.751926 28096 leader_election.cc:288] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:41.752079 28101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:41.754940 28101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [term 1 LEADER]: Becoming Leader. State: Replica: 697dc1d624e64948868a3bcbbfba4833, State: Running, Role: LEADER
I20260806 19:42:41.755055 28096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:41.755578 28101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697dc1d624e64948868a3bcbbfba4833" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42809 } }
I20260806 19:42:41.758636 28102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "697dc1d624e64948868a3bcbbfba4833" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697dc1d624e64948868a3bcbbfba4833" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42809 } } }
I20260806 19:42:41.759001 28102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:41.759634 28103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 697dc1d624e64948868a3bcbbfba4833. Latest consensus state: current_term: 1 leader_uuid: "697dc1d624e64948868a3bcbbfba4833" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697dc1d624e64948868a3bcbbfba4833" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42809 } } }
I20260806 19:42:41.759975 28103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:41.760134 28110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:41.765841 28110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:41.777179 28110 catalog_manager.cc:1347] Generated new cluster ID: 8c33aefef72f4e7d95052d7a7c52cbdc
I20260806 19:42:41.777357 28110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:41.787611 28110 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:41.789228 28110 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:41.800349 28110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 697dc1d624e64948868a3bcbbfba4833: Generated new TSK 0
I20260806 19:42:41.801496 28110 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:42:41.873750 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42809
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:42.229640 28122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:42.229959 28122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:42.230155 28122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:42.237879 28122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:42.238020 28122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:42.238237 28122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:42:42.248322 28122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:42.249786 28122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:42.251333 28122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:42.266054 28127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:42.266022 28128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:42.265996 28130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:42.267035 28122 server_base.cc:1034] running on GCE node
I20260806 19:42:42.267787 28122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:42.268958 28122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:42.270205 28122 hybrid_clock.cc:648] HybridClock initialized: now 1786045362270139 us; error 47 us; skew 500 ppm
I20260806 19:42:42.270578 28122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:42 dist-test-slave-77v9 krb5kdc[28015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045362, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:42.274087 28122 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:42:42.275723 28122 webserver.cc:460] Webserver started at http://127.21.202.65:34521/ using document root <none> and password file <none>
I20260806 19:42:42.276491 28122 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:42.276655 28122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:42.277065 28122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:42.280474 28122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "fd1f056333e44de88096ef3da894b59f"
format_stamp: "Formatted at 2026-08-06 19:42:42 on dist-test-slave-77v9"
server_key: "14d8922230e35661fbd4a551fe441c08"
server_key_iv: "8f680970ee0d1b67e4c8b42d6d93f657"
server_key_version: "encryptionkey@0"
I20260806 19:42:42.281478 28122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "fd1f056333e44de88096ef3da894b59f"
format_stamp: "Formatted at 2026-08-06 19:42:42 on dist-test-slave-77v9"
server_key: "14d8922230e35661fbd4a551fe441c08"
server_key_iv: "8f680970ee0d1b67e4c8b42d6d93f657"
server_key_version: "encryptionkey@0"
I20260806 19:42:42.287243 28122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:42:42.291060 28137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:42.292507 28122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:42:42.292704 28122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "fd1f056333e44de88096ef3da894b59f"
format_stamp: "Formatted at 2026-08-06 19:42:42 on dist-test-slave-77v9"
server_key: "14d8922230e35661fbd4a551fe441c08"
server_key_iv: "8f680970ee0d1b67e4c8b42d6d93f657"
server_key_version: "encryptionkey@0"
I20260806 19:42:42.292984 28122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:42.315618 28122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:42.320874 28122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:42.321214 28122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:42.322319 28122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:42.323823 28122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:42.323945 28122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:42.324118 28122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:42.324199 28122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:42.362923 28122 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:45787
I20260806 19:42:42.362973 28250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:45787 every 8 connection(s)
I20260806 19:42:42.364432 28122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:42:42.370366 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 28122
I20260806 19:42:42.370618 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:42:42.371337 22313 external_mini_cluster.cc:1428] Setting key 3ef2b8081ac97c4bd1fe8f7bd46e3622
Aug 06 19:42:42 dist-test-slave-77v9 krb5kdc[28015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045362, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:42.388376 28253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:42.367204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:60441 (local address 127.21.202.126:42809)
0806 19:42:42.367747 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:42.367752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:42.368553 (+ 801us) server_negotiation.cc:407] Connection header received
0806 19:42:42.370685 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:42.370691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:42.370798 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:42.370925 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:42.372776 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:42.374041 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:42.375774 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:42.376588 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:42.376753 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:42.381064 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:42.381074 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:42.381083 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:42.381134 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:42.384357 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:42.385503 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:42.385508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:42.385511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:42.385670 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:42.386261 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:42.386265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:42.386267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:42.386600 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:42.386729 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:42.387864 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0806 19:42:42.388065 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":288,"threads_started":1}
I20260806 19:42:42.390197 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:42.367599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42809 (local address 127.21.202.65:60441)
0806 19:42:42.368381 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:42.368421 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:42.370353 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:42.371176 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:42.371184 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:42.371769 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:42.372581 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:42.372615 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:42.374233 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:42.374237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:42.375609 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:42.375622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:42.376955 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:42.376963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:42.377190 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:42.377979 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:42.378006 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:42.380731 (+ 2725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:42.384531 (+ 3800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:42.384545 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:42.384567 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:42.385342 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:42.385814 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:42.385823 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:42.385830 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:42.386110 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:42.386754 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:42.386765 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:42.387306 (+ 541us) client_negotiation.cc:769] Sending connection context
0806 19:42:42.387683 (+ 377us) client_negotiation.cc:240] Negotiation successful
0806 19:42:42.388371 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":316,"threads_started":1}
I20260806 19:42:42.392385 28251 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42809
I20260806 19:42:42.392769 28251 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:42.393868 28251 heartbeater.cc:511] Master 127.21.202.126:42809 requested a full tablet report, sending...
I20260806 19:42:42.396781 28061 ts_manager.cc:194] Registered new tserver with Master: fd1f056333e44de88096ef3da894b59f (127.21.202.65:45787)
I20260806 19:42:42.398749 28061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:60441
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:42:42.432745 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42809
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:42.790668 28257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:42.790998 28257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:42.791157 28257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:42.798892 28257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:42.799016 28257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:42.799187 28257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:42:42.809763 28257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:42.811141 28257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:42.812530 28257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:42.828081 28265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:42.828066 28263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:42.828326 28262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:42.828796 28257 server_base.cc:1034] running on GCE node
I20260806 19:42:42.829581 28257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:42.830579 28257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:42.831820 28257 hybrid_clock.cc:648] HybridClock initialized: now 1786045362831770 us; error 45 us; skew 500 ppm
I20260806 19:42:42.832157 28257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:42 dist-test-slave-77v9 krb5kdc[28015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045362, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:42.835558 28257 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:42:42.837435 28257 webserver.cc:460] Webserver started at http://127.21.202.66:33235/ using document root <none> and password file <none>
I20260806 19:42:42.838200 28257 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:42.838370 28257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:42.838723 28257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:42.842042 28257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "8e07ffeb5263469e9436fa4487ef4e19"
format_stamp: "Formatted at 2026-08-06 19:42:42 on dist-test-slave-77v9"
server_key: "f7fbf5fd1362e744cf21f06985cb27be"
server_key_iv: "2130e8c0ebca49a10666e4fb7a95e025"
server_key_version: "encryptionkey@0"
I20260806 19:42:42.842926 28257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "8e07ffeb5263469e9436fa4487ef4e19"
format_stamp: "Formatted at 2026-08-06 19:42:42 on dist-test-slave-77v9"
server_key: "f7fbf5fd1362e744cf21f06985cb27be"
server_key_iv: "2130e8c0ebca49a10666e4fb7a95e025"
server_key_version: "encryptionkey@0"
I20260806 19:42:42.848553 28257 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 19:42:42.852371 28272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:42.854004 28257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:42.854167 28257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "8e07ffeb5263469e9436fa4487ef4e19"
format_stamp: "Formatted at 2026-08-06 19:42:42 on dist-test-slave-77v9"
server_key: "f7fbf5fd1362e744cf21f06985cb27be"
server_key_iv: "2130e8c0ebca49a10666e4fb7a95e025"
server_key_version: "encryptionkey@0"
I20260806 19:42:42.854398 28257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:42.885129 28257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:42.889951 28257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:42.890228 28257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:42.891340 28257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:42.892791 28257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:42.892889 28257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 19:42:42.892979 28257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:42.893029 28257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:42.930967 28257 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:46517
I20260806 19:42:42.931078 28386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:46517 every 8 connection(s)
I20260806 19:42:42.932289 28257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:42:42.939615 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 28257
I20260806 19:42:42.939882 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:42:42.940398 22313 external_mini_cluster.cc:1428] Setting key ddd1dfd73948cd6ee50bda43afe10d94
Aug 06 19:42:42 dist-test-slave-77v9 krb5kdc[28015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045362, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:42.954725 28389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:42.935270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:41005 (local address 127.21.202.126:42809)
0806 19:42:42.935765 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:42.935769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:42.936848 (+ 1079us) server_negotiation.cc:407] Connection header received
0806 19:42:42.938957 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:42.938964 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:42.939063 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:42.939201 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:42.941097 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:42.941945 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:42.943671 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:42.944251 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:42.944405 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:42.948555 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:42.948565 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:42.948569 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:42.948619 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:42.951603 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:42.952346 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:42.952352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:42.952356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:42.952495 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:42.952962 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:42.952967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:42.952970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:42.953256 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:42.953425 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:42.954107 (+ 682us) server_negotiation.cc:299] Negotiation successful
0806 19:42:42.954323 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":266,"threads_started":1}
I20260806 19:42:42.955983 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:42.935822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42809 (local address 127.21.202.66:41005)
0806 19:42:42.936623 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:42.936676 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:42.938648 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:42.939460 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:42.939469 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:42.940177 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:42.940913 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:42.940935 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:42.942121 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:42.942132 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:42.943510 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:42.943529 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:42.944507 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:42.944517 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:42.944799 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:42.945651 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:42.945716 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:42.948283 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:42.951745 (+ 3462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:42.951755 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:42.951773 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:42.952214 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:42.952609 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:42.952614 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:42.952617 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:42.952827 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:42.953384 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:42.953392 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:42.953729 (+ 337us) client_negotiation.cc:769] Sending connection context
0806 19:42:42.953999 (+ 270us) client_negotiation.cc:240] Negotiation successful
0806 19:42:42.954538 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":337,"threads_started":1}
I20260806 19:42:42.957856 28387 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42809
I20260806 19:42:42.958228 28387 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:42.959254 28387 heartbeater.cc:511] Master 127.21.202.126:42809 requested a full tablet report, sending...
I20260806 19:42:42.961179 28061 ts_manager.cc:194] Registered new tserver with Master: 8e07ffeb5263469e9436fa4487ef4e19 (127.21.202.66:46517)
I20260806 19:42:42.962637 28061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:41005
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:42:43.001492 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42809
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:43.337150 28393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:43.337463 28393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:43.337636 28393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:43.344866 28393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:43.344977 28393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:43.345220 28393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:42:43.355841 28393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:43.357319 28393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:43.358692 28393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:43.372457 28398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:43.372572 28399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:43.372478 28401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:43.373831 28393 server_base.cc:1034] running on GCE node
I20260806 19:42:43.374543 28393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:43.375614 28393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:43.376914 28393 hybrid_clock.cc:648] HybridClock initialized: now 1786045363376883 us; error 47 us; skew 500 ppm
I20260806 19:42:43.377257 28393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:43.380669 28393 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:42:43.382270 28393 webserver.cc:460] Webserver started at http://127.21.202.67:39511/ using document root <none> and password file <none>
I20260806 19:42:43.383015 28393 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:43.383198 28393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:43.383533 28393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:43.387019 28393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "3da3747fff7443ec96d3b07888bb6b9a"
format_stamp: "Formatted at 2026-08-06 19:42:43 on dist-test-slave-77v9"
server_key: "bf8f8960924cc7cfab2e909184365867"
server_key_iv: "812d1601d7448f5d60bb5b2579b6bc29"
server_key_version: "encryptionkey@0"
I20260806 19:42:43.387892 28393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "3da3747fff7443ec96d3b07888bb6b9a"
format_stamp: "Formatted at 2026-08-06 19:42:43 on dist-test-slave-77v9"
server_key: "bf8f8960924cc7cfab2e909184365867"
server_key_iv: "812d1601d7448f5d60bb5b2579b6bc29"
server_key_version: "encryptionkey@0"
I20260806 19:42:43.393512 28393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 19:42:43.397320 28408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:43.398945 28393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:42:43.399152 28393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "3da3747fff7443ec96d3b07888bb6b9a"
format_stamp: "Formatted at 2026-08-06 19:42:43 on dist-test-slave-77v9"
server_key: "bf8f8960924cc7cfab2e909184365867"
server_key_iv: "812d1601d7448f5d60bb5b2579b6bc29"
server_key_version: "encryptionkey@0"
I20260806 19:42:43.399387 28393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:43.402438 28251 heartbeater.cc:503] Master 127.21.202.126:42809 was elected leader, sending a full tablet report...
I20260806 19:42:43.415071 28393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:43.419456 28393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:43.419790 28393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:43.420883 28393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:43.422385 28393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:43.422493 28393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:43.422642 28393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:43.422721 28393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:43.464493 28393 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:43853
I20260806 19:42:43.464612 28520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:43853 every 8 connection(s)
I20260806 19:42:43.466019 28393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:42:43.467847 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 28393
I20260806 19:42:43.468103 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:42:43.468689 22313 external_mini_cluster.cc:1428] Setting key 95a5a34ab866ede58104babbae1c724d
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:42:43.487674 28523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:43.468894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:43631 (local address 127.21.202.126:42809)
0806 19:42:43.469430 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:43.469441 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:43.470389 (+ 948us) server_negotiation.cc:407] Connection header received
0806 19:42:43.472828 (+ 2439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:43.472833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:43.472925 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:43.473054 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:43.474559 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.475365 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.476963 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.477421 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.477535 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:43.481156 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:43.481162 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:43.481165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:43.481196 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:43.484483 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.485399 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.485403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.485405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.485529 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.485959 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.485963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.485965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.486245 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:43.486387 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:43.487137 (+ 750us) server_negotiation.cc:299] Negotiation successful
0806 19:42:43.487331 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":261,"threads_started":1}
I20260806 19:42:43.488442 28524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:43.469503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42809 (local address 127.21.202.67:43631)
0806 19:42:43.470206 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:43.470244 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:43.472527 (+ 2283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:43.473298 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:43.473306 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:43.473813 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:43.474391 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:43.474409 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.475523 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.475528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:43.476756 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:43.476769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.477629 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.477634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:43.477832 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:43.478570 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:43.478593 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:43.480925 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:43.484609 (+ 3684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:43.484616 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:43.484629 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:43.485274 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:43.485620 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:43.485623 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:43.485625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:43.485797 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:43.486360 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:43.486366 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:43.486709 (+ 343us) client_negotiation.cc:769] Sending connection context
0806 19:42:43.486972 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 19:42:43.487359 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":427,"threads_started":1}
I20260806 19:42:43.489909 28521 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42809
I20260806 19:42:43.490254 28521 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:43.491000 28521 heartbeater.cc:511] Master 127.21.202.126:42809 requested a full tablet report, sending...
I20260806 19:42:43.492430 28061 ts_manager.cc:194] Registered new tserver with Master: 3da3747fff7443ec96d3b07888bb6b9a (127.21.202.67:43853)
I20260806 19:42:43.493366 28061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:43631
I20260806 19:42:43.497853 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/test-admin.keytab.
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:43.530802 22313 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 19:42:43.536875 28529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:42:43.534038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44936 (local address 127.21.202.65:45787)
0806 19:42:43.534765 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:43.534773 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:43.534819 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:42:43.534890 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:43.534897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:43.535132 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:43.535281 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:43.536424 (+ 1143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44936: BlockingRecv error: recv got EOF from 127.0.0.1:44936 (error 108)
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":413,"threads_started":1}
W20260806 19:42:43.537376 22313 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/krb5cc' of type FILE
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:43.539167 22313 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:43.540743 22313 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:42:43.562145 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:43.542612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44950 (local address 127.21.202.65:45787)
0806 19:42:43.542846 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:43.542853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:43.543150 (+ 297us) server_negotiation.cc:407] Connection header received
0806 19:42:43.543370 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:43.543375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:43.543453 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:43.543562 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:43.544570 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.545616 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.546969 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.547576 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.547728 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:43.555527 (+ 7799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:43.555543 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:43.555559 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:43.555604 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:43.558958 (+ 3354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.559770 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.559778 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.559786 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.559941 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.560354 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.560361 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.560365 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.560763 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:43.561146 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:43.561523 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 19:42:43.561816 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260806 19:42:43.569201 28529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:42:43.567348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44958 (local address 127.21.202.65:45787)
0806 19:42:43.567578 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:43.567583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:43.567758 (+ 175us) server_negotiation.cc:407] Connection header received
0806 19:42:43.567885 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:43.567888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:43.567946 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:43.568026 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:43.568898 (+ 872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44958: BlockingRecv error: recv got EOF from 127.0.0.1:44958 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 19:42:43.569262 22313 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/krb5cc' of type FILE
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:43.570842 22313 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:43.572165 22313 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:42:43.587262 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:43.573843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44970 (local address 127.21.202.65:45787)
0806 19:42:43.574066 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:43.574072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:43.574241 (+ 169us) server_negotiation.cc:407] Connection header received
0806 19:42:43.574451 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:43.574456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:43.574528 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:43.574633 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:43.575597 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.576324 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.577675 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.578136 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.578269 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:43.582005 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:43.582010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:43.582014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:43.582060 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:43.584605 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.585304 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.585312 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.585317 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.585455 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.585857 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.585862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.585867 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.586140 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:43.586443 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:43.586753 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 19:42:43.586977 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 19:42:43.592357 28529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:42:43.590286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44976 (local address 127.21.202.65:45787)
0806 19:42:43.590526 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:43.590535 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:43.590921 (+ 386us) server_negotiation.cc:407] Connection header received
0806 19:42:43.591125 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:43.591131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:43.591219 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:43.591337 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:43.592102 (+ 765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44976: BlockingRecv error: recv got EOF from 127.0.0.1:44976 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 19:42:43.592599 22313 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786045310306758-22313-0/krb5kdc/krb5cc' of type FILE
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:43.594415 22313 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:43.595777 22313 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:42:43.611191 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:43.598044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44992 (local address 127.21.202.65:45787)
0806 19:42:43.598260 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:43.598265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:43.598282 (+ 17us) server_negotiation.cc:407] Connection header received
0806 19:42:43.598330 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:43.598334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:43.598392 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:43.598473 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:43.599479 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.600222 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.601604 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.602059 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.602167 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:43.606005 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:43.606010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:43.606013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:43.606048 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:43.608782 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.609443 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.609447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.609450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.609606 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.609941 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.609944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.609946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.610204 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:43.610449 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:43.610639 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 19:42:43.610847 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 19:42:43.613016 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 28122
I20260806 19:42:43.625828 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:43.613741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45000 (local address 127.21.202.65:45787)
0806 19:42:43.613902 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:43.613906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:43.614413 (+ 507us) server_negotiation.cc:407] Connection header received
0806 19:42:43.614515 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:43.614519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:43.614582 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:43.614673 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:43.615582 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.616609 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.617859 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.618372 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.618478 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:43.620469 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:43.620474 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:43.620477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:43.620508 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:43.623117 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.623758 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.623763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.623765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.623945 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.624361 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.624370 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.624373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.624731 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:43.625032 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:43.625382 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 19:42:43.625561 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 19:42:43.626562 28246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:43.712925 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 28122
I20260806 19:42:43.731510 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 28257
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:42:43.748400 28542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:43.732551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41836 (local address 127.21.202.66:46517)
0806 19:42:43.733259 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:43.733266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:43.733295 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:42:43.733365 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:43.733372 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:43.733592 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:43.733731 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:43.734756 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.735947 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.737661 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.738333 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.738464 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:43.742445 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:43.742457 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:43.742471 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:43.742506 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:43.745477 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.746226 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.746233 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.746235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.746353 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.746802 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.746805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.746807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.747145 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:43.747379 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:43.747758 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 19:42:43.748071 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":425,"threads_started":1}
I20260806 19:42:43.749886 28382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:43.835772 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 28257
I20260806 19:42:43.852931 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 28393
Aug 06 19:42:43 dist-test-slave-77v9 krb5kdc[28015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:42:43.870061 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:43.853963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53508 (local address 127.21.202.67:43853)
0806 19:42:43.854820 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:43.854829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:43.854860 (+ 31us) server_negotiation.cc:407] Connection header received
0806 19:42:43.854940 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:43.854947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:43.855198 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:43.855376 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:43.856571 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.857803 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.859108 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:43.859689 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:43.859833 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:43.864220 (+ 4387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:43.864231 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:43.864247 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:43.864285 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:43.867437 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.868119 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.868126 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.868128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.868244 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:43.868581 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:43.868585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:43.868587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:43.868983 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:43.869221 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:43.869463 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 19:42:43.869716 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":515,"threads_started":1}
I20260806 19:42:43.871433 28516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:43.955792 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 28393
I20260806 19:42:43.974980 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 28028
I20260806 19:42:43.976269 28091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:44.088217 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 28028
2026-08-06T19:42:44Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4270 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:42:44 dist-test-slave-77v9 krb5kdc[28549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:42:44 dist-test-slave-77v9 krb5kdc[28549](info): set up 2 sockets
Aug 06 19:42:44 dist-test-slave-77v9 krb5kdc[28549](info): commencing operation
krb5kdc: starting...
W20260806 19:42:45.270238 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:42:45 dist-test-slave-77v9 krb5kdc[28549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:42:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.126.keytab.
Entry for principal oryx/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.126.keytab.
I20260806 19:42:45.545725 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:45677
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.126.keytab
--principal=oryx/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:35153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:45677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:45.894696 28562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:45.895051 28562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:45.895212 28562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:45.902391 28562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:45.902494 28562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:45.902587 28562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:45.902698 28562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:45.902762 28562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:42:45.912639 28562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:35153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:45677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.126.keytab
--principal=oryx/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:45677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:45.914220 28562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:45.915634 28562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:45.927802 28570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:45.927906 28568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:45.928138 28567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:45.929075 28562 server_base.cc:1034] running on GCE node
I20260806 19:42:45.929939 28562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:45.931524 28562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:45.932790 28562 hybrid_clock.cc:648] HybridClock initialized: now 1786045365932740 us; error 34 us; skew 500 ppm
I20260806 19:42:45.933224 28562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:45 dist-test-slave-77v9 krb5kdc[28549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045365, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:45.936427 28562 init.cc:377] Logged in from keytab as oryx/127.21.202.126@KRBTEST.COM (short username oryx)
I20260806 19:42:45.938035 28562 webserver.cc:460] Webserver started at http://127.21.202.126:37791/ using document root <none> and password file <none>
I20260806 19:42:45.938760 28562 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:45.938922 28562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:45.939256 28562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:45.942461 28562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "ec97e781650a49da8405e8eff3b5242e"
format_stamp: "Formatted at 2026-08-06 19:42:45 on dist-test-slave-77v9"
server_key: "a40c661ee63f6f37641f58c750ba4ff5"
server_key_iv: "dc8302c80e3c392234edf3a3478b25cb"
server_key_version: "encryptionkey@0"
I20260806 19:42:45.943266 28562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "ec97e781650a49da8405e8eff3b5242e"
format_stamp: "Formatted at 2026-08-06 19:42:45 on dist-test-slave-77v9"
server_key: "a40c661ee63f6f37641f58c750ba4ff5"
server_key_iv: "dc8302c80e3c392234edf3a3478b25cb"
server_key_version: "encryptionkey@0"
I20260806 19:42:45.948946 28562 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 19:42:45.952584 28577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:45.954119 28562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:45.954339 28562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "ec97e781650a49da8405e8eff3b5242e"
format_stamp: "Formatted at 2026-08-06 19:42:45 on dist-test-slave-77v9"
server_key: "a40c661ee63f6f37641f58c750ba4ff5"
server_key_iv: "dc8302c80e3c392234edf3a3478b25cb"
server_key_version: "encryptionkey@0"
I20260806 19:42:45.954553 28562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:46.003301 28562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:46.007990 28562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:46.008324 28562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:46.028915 28562 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:45677
I20260806 19:42:46.028936 28628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:45677 every 8 connection(s)
I20260806 19:42:46.030364 28562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:46.032897 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 28562
I20260806 19:42:46.033171 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:46.034036 22313 external_mini_cluster.cc:1428] Setting key 8e264c34cc15451d4e3572ed7a9065df
I20260806 19:42:46.035343 28629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 19:42:46 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
I20260806 19:42:46.050354 28629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e: Bootstrap starting.
I20260806 19:42:46.053977 28629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:46.055029 28629 log.cc:824] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:46.058148 28629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e: No bootstrap required, opened a new log
I20260806 19:42:46.058249 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:46.036277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51258 (local address 127.21.202.126:45677)
0806 19:42:46.037114 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:46.037125 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:46.037167 (+ 42us) server_negotiation.cc:407] Connection header received
0806 19:42:46.038717 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:46.038738 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:46.039132 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:46.039462 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:46.041374 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:46.042775 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:46.044531 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:46.045345 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:46.045490 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:46.050384 (+ 4894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:46.050392 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:46.050408 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:46.050452 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:46.053831 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:46.054679 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:46.054687 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:46.054693 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:46.054874 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:46.055297 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:46.055302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:46.055307 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:46.055857 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:46.056211 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:46.056652 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 19:42:46.057133 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":360,"threads_started":1}
I20260806 19:42:46.066198 28629 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } }
I20260806 19:42:46.066613 28629 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:46.066722 28629 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec97e781650a49da8405e8eff3b5242e, State: Initialized, Role: FOLLOWER
I20260806 19:42:46.067355 28629 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } }
I20260806 19:42:46.067593 28629 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:46.067749 28629 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:46.067952 28629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:46.070221 28629 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } }
I20260806 19:42:46.070765 28629 leader_election.cc:302] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec97e781650a49da8405e8eff3b5242e; no voters:
I20260806 19:42:46.071406 28629 leader_election.cc:288] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:46.071553 28634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:46.072253 28634 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [term 1 LEADER]: Becoming Leader. State: Replica: ec97e781650a49da8405e8eff3b5242e, State: Running, Role: LEADER
I20260806 19:42:46.072921 28634 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } }
I20260806 19:42:46.073625 28629 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:46.075749 28636 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec97e781650a49da8405e8eff3b5242e. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } }
I20260806 19:42:46.075897 28635 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } }
I20260806 19:42:46.076191 28636 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:46.076232 28635 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:46.077428 28641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:46.084039 28641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:46.093788 28641 catalog_manager.cc:1347] Generated new cluster ID: 01055745776b4fecaad51f018b19e9d3
I20260806 19:42:46.093930 28641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:46.105041 28641 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:46.106416 28641 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:46.113497 28641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e: Generated new TSK 0
I20260806 19:42:46.114379 28641 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab.
Entry for principal oryx/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab.
I20260806 19:42:46.185218 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:45677
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab
--principal=oryx/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:35153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:46.532507 28655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:46.532843 28655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:46.532979 28655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:46.541520 28655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:46.541648 28655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:46.541847 28655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:42:46.552748 28655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:35153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab
--principal=oryx/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:45677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:46.554258 28655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:46.555707 28655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:46.571678 28664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:46.571818 28661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:46.571818 28660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:46.572497 28655 server_base.cc:1034] running on GCE node
I20260806 19:42:46.573218 28655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:46.574520 28655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:46.576035 28655 hybrid_clock.cc:648] HybridClock initialized: now 1786045366575882 us; error 84 us; skew 500 ppm
I20260806 19:42:46.576484 28655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:46 dist-test-slave-77v9 krb5kdc[28549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045366, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:46.580653 28655 init.cc:377] Logged in from keytab as oryx/127.21.202.65@KRBTEST.COM (short username oryx)
I20260806 19:42:46.582724 28655 webserver.cc:460] Webserver started at http://127.21.202.65:33463/ using document root <none> and password file <none>
I20260806 19:42:46.583760 28655 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:46.583949 28655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:46.584343 28655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:46.587960 28655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "d64104c8ac3b4b4fb0f11c7d035d575c"
format_stamp: "Formatted at 2026-08-06 19:42:46 on dist-test-slave-77v9"
server_key: "0e3484d57efdd78b4378c49d9a95c761"
server_key_iv: "c1b46bd63db657b397f82dc9964128f2"
server_key_version: "encryptionkey@0"
I20260806 19:42:46.588938 28655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "d64104c8ac3b4b4fb0f11c7d035d575c"
format_stamp: "Formatted at 2026-08-06 19:42:46 on dist-test-slave-77v9"
server_key: "0e3484d57efdd78b4378c49d9a95c761"
server_key_iv: "c1b46bd63db657b397f82dc9964128f2"
server_key_version: "encryptionkey@0"
I20260806 19:42:46.594933 28655 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 19:42:46.599434 28671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:46.601432 28655 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 19:42:46.601621 28655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "d64104c8ac3b4b4fb0f11c7d035d575c"
format_stamp: "Formatted at 2026-08-06 19:42:46 on dist-test-slave-77v9"
server_key: "0e3484d57efdd78b4378c49d9a95c761"
server_key_iv: "c1b46bd63db657b397f82dc9964128f2"
server_key_version: "encryptionkey@0"
I20260806 19:42:46.601845 28655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:46.615370 28655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:46.619948 28655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:46.620271 28655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:46.621356 28655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:46.622879 28655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:46.622958 28655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:46.623114 28655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:46.623178 28655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:46.659313 28655 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:36007
I20260806 19:42:46.659358 28785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:36007 every 8 connection(s)
I20260806 19:42:46.660635 28655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:42:46.663288 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 28655
I20260806 19:42:46.663543 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:42:46.664420 22313 external_mini_cluster.cc:1428] Setting key 241eaeff54d7fda16952eeb7b0bfed4b
Aug 06 19:42:46 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045366, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
I20260806 19:42:46.686441 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:46.663362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:41573 (local address 127.21.202.126:45677)
0806 19:42:46.665732 (+ 2370us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:46.665738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:46.665758 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:42:46.667745 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:46.667749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:46.667813 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:46.667919 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:46.670085 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:46.670826 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:46.673386 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:46.673791 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:46.673925 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:46.678349 (+ 4424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:46.678360 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:46.678368 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:46.678424 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:46.682545 (+ 4121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:46.683377 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:46.683386 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:46.683392 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:46.683562 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:46.684110 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:46.684118 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:46.684124 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:46.684564 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:46.684746 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:46.685624 (+ 878us) server_negotiation.cc:299] Negotiation successful
0806 19:42:46.685904 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2233,"thread_start_us":2120,"threads_started":1}
I20260806 19:42:46.687073 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:46.664035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.65:41573)
0806 19:42:46.665157 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:46.665219 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:46.667438 (+ 2219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:46.668196 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:46.668210 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:46.669059 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:46.669892 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:46.669919 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:46.671001 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:46.671009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:46.673207 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:46.673226 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:46.674027 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:46.674037 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:46.674380 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:46.675298 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:46.675333 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:46.678030 (+ 2697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:46.682719 (+ 4689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:46.682726 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:46.682741 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:46.683213 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:46.683733 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:46.683736 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:46.683739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:46.683948 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:46.684704 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:46.684709 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:46.685179 (+ 470us) client_negotiation.cc:769] Sending connection context
0806 19:42:46.685420 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 19:42:46.685845 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":440,"threads_started":1}
I20260806 19:42:46.688875 28786 heartbeater.cc:347] Connected to a master server at 127.21.202.126:45677
I20260806 19:42:46.689206 28786 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:46.689970 28786 heartbeater.cc:511] Master 127.21.202.126:45677 requested a full tablet report, sending...
I20260806 19:42:46.692157 28594 ts_manager.cc:194] Registered new tserver with Master: d64104c8ac3b4b4fb0f11c7d035d575c (127.21.202.65:36007)
WARNING: no policy specified for oryx/127.21.202.66@KRBTEST.COM; defaulting to no policy
I20260806 19:42:46.693789 28594 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:41573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab.
Entry for principal oryx/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab.
I20260806 19:42:46.729847 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:45677
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab
--principal=oryx/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:35153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:47.096410 28792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:47.096715 28792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:47.096962 28792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:47.105245 28792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:47.105391 28792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:47.105518 28792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:42:47.116252 28792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:35153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab
--principal=oryx/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:45677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:47.117722 28792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:47.119151 28792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:47.133816 28798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:47.133817 28800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:47.134495 28797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:47.135396 28792 server_base.cc:1034] running on GCE node
I20260806 19:42:47.136229 28792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:47.137495 28792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:47.138803 28792 hybrid_clock.cc:648] HybridClock initialized: now 1786045367138747 us; error 60 us; skew 500 ppm
I20260806 19:42:47.139194 28792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:47 dist-test-slave-77v9 krb5kdc[28549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045367, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:47.142889 28792 init.cc:377] Logged in from keytab as oryx/127.21.202.66@KRBTEST.COM (short username oryx)
I20260806 19:42:47.144601 28792 webserver.cc:460] Webserver started at http://127.21.202.66:34485/ using document root <none> and password file <none>
I20260806 19:42:47.145439 28792 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:47.145598 28792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:47.145932 28792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:47.149101 28792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "d3bffa83f0eb452ab48524820f9e1aea"
format_stamp: "Formatted at 2026-08-06 19:42:47 on dist-test-slave-77v9"
server_key: "a9e88475725d964a2fa081072fc0bf48"
server_key_iv: "049998c93c76b38b5fd0a4c45f6bfcf6"
server_key_version: "encryptionkey@0"
I20260806 19:42:47.149966 28792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "d3bffa83f0eb452ab48524820f9e1aea"
format_stamp: "Formatted at 2026-08-06 19:42:47 on dist-test-slave-77v9"
server_key: "a9e88475725d964a2fa081072fc0bf48"
server_key_iv: "049998c93c76b38b5fd0a4c45f6bfcf6"
server_key_version: "encryptionkey@0"
I20260806 19:42:47.155695 28792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:42:47.159760 28807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:47.161381 28792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:47.161578 28792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "d3bffa83f0eb452ab48524820f9e1aea"
format_stamp: "Formatted at 2026-08-06 19:42:47 on dist-test-slave-77v9"
server_key: "a9e88475725d964a2fa081072fc0bf48"
server_key_iv: "049998c93c76b38b5fd0a4c45f6bfcf6"
server_key_version: "encryptionkey@0"
I20260806 19:42:47.161808 28792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:47.191243 28792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:47.196149 28792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:47.196491 28792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:47.197589 28792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:47.199015 28792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:47.199121 28792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:47.199273 28792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:47.199347 28792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:47.236549 28792 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:33619
I20260806 19:42:47.236593 28920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:33619 every 8 connection(s)
I20260806 19:42:47.238091 28792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:42:47.244144 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 28792
I20260806 19:42:47.244403 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:42:47.244920 22313 external_mini_cluster.cc:1428] Setting key 83c2ae5f5877bc60058aab2d05ea9562
Aug 06 19:42:47 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045367, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
I20260806 19:42:47.260215 28923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:47.240776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:41795 (local address 127.21.202.126:45677)
0806 19:42:47.241251 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:47.241256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:47.242381 (+ 1125us) server_negotiation.cc:407] Connection header received
0806 19:42:47.244252 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:47.244258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:47.244354 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:47.244493 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:47.246635 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:47.247427 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:47.249059 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:47.249499 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:47.249637 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:47.253577 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:47.253582 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:47.253585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:47.253619 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:47.256416 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:47.257289 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:47.257293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:47.257296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:47.257445 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:47.257907 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:47.257912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:47.257915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:47.258299 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:47.258464 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:47.259363 (+ 899us) server_negotiation.cc:299] Negotiation successful
0806 19:42:47.259647 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":217,"threads_started":1}
I20260806 19:42:47.260881 28924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:47.241377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.66:41795)
0806 19:42:47.242209 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:47.242256 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:47.243967 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:47.244732 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:47.244741 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:47.245717 (+ 976us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:47.246435 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:47.246466 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:47.247595 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:47.247605 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:47.248915 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:47.248928 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:47.249751 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:47.249758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:47.250002 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:47.250742 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:47.250765 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:47.253308 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:47.256597 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:47.256606 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:47.256626 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:47.257147 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:47.257564 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:47.257569 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:47.257573 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:47.257775 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:47.258433 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:47.258442 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:47.258798 (+ 356us) client_negotiation.cc:769] Sending connection context
0806 19:42:47.259101 (+ 303us) client_negotiation.cc:240] Negotiation successful
0806 19:42:47.259636 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":302,"threads_started":1}
I20260806 19:42:47.262581 28921 heartbeater.cc:347] Connected to a master server at 127.21.202.126:45677
I20260806 19:42:47.262985 28921 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:47.263969 28921 heartbeater.cc:511] Master 127.21.202.126:45677 requested a full tablet report, sending...
I20260806 19:42:47.265930 28594 ts_manager.cc:194] Registered new tserver with Master: d3bffa83f0eb452ab48524820f9e1aea (127.21.202.66:33619)
I20260806 19:42:47.266964 28594 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:41795
WARNING: no policy specified for oryx/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab.
Entry for principal oryx/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab.
I20260806 19:42:47.305835 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:45677
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab
--principal=oryx/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:35153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:47.646339 28927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:47.646704 28927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:47.646893 28927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:47.654150 28927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:47.654304 28927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:47.654505 28927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:42:47.667176 28927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:35153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab
--principal=oryx/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:45677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:47.668426 28927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:47.669826 28927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:47.683864 28935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:47.683955 28932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:47.683955 28933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:47.685127 28927 server_base.cc:1034] running on GCE node
I20260806 19:42:47.685801 28927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:47.686847 28927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:47.688086 28927 hybrid_clock.cc:648] HybridClock initialized: now 1786045367688012 us; error 55 us; skew 500 ppm
I20260806 19:42:47.688484 28927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:47 dist-test-slave-77v9 krb5kdc[28549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045367, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:47.692111 28927 init.cc:377] Logged in from keytab as oryx/127.21.202.67@KRBTEST.COM (short username oryx)
I20260806 19:42:47.693910 28927 webserver.cc:460] Webserver started at http://127.21.202.67:43925/ using document root <none> and password file <none>
I20260806 19:42:47.694674 28927 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:47.694829 28927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:47.695186 28927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:47.697031 28786 heartbeater.cc:503] Master 127.21.202.126:45677 was elected leader, sending a full tablet report...
I20260806 19:42:47.699227 28927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "0a5049e41f514390804e74e692bc90bf"
format_stamp: "Formatted at 2026-08-06 19:42:47 on dist-test-slave-77v9"
server_key: "2a926aa3967a6b505a188ab057f022e4"
server_key_iv: "a0362585e8b450738c2c7ee80cdf2b7f"
server_key_version: "encryptionkey@0"
I20260806 19:42:47.700212 28927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "0a5049e41f514390804e74e692bc90bf"
format_stamp: "Formatted at 2026-08-06 19:42:47 on dist-test-slave-77v9"
server_key: "2a926aa3967a6b505a188ab057f022e4"
server_key_iv: "a0362585e8b450738c2c7ee80cdf2b7f"
server_key_version: "encryptionkey@0"
I20260806 19:42:47.705993 28927 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 19:42:47.709751 28942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:47.711261 28927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:42:47.711474 28927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "0a5049e41f514390804e74e692bc90bf"
format_stamp: "Formatted at 2026-08-06 19:42:47 on dist-test-slave-77v9"
server_key: "2a926aa3967a6b505a188ab057f022e4"
server_key_iv: "a0362585e8b450738c2c7ee80cdf2b7f"
server_key_version: "encryptionkey@0"
I20260806 19:42:47.711694 28927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:47.732311 28927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:47.737138 28927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:47.737484 28927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:47.738595 28927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:47.740183 28927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:47.740298 28927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:47.740471 28927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:47.740553 28927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:47.780450 28927 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:42981
I20260806 19:42:47.780486 29056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:42981 every 8 connection(s)
I20260806 19:42:47.781829 28927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:42:47.791981 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 28927
I20260806 19:42:47.792224 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:42:47.792690 22313 external_mini_cluster.cc:1428] Setting key 00b84089bc50417a7032a09a7dda08ce
Aug 06 19:42:47 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045367, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
I20260806 19:42:47.803983 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:47.784631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:40105 (local address 127.21.202.126:45677)
0806 19:42:47.785282 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:47.785287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:47.786442 (+ 1155us) server_negotiation.cc:407] Connection header received
0806 19:42:47.788521 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:47.788527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:47.788657 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:47.788804 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:47.790481 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:47.791404 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:47.793391 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:47.794057 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:47.794205 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:47.798508 (+ 4303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:47.798513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:47.798516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:47.798551 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:47.801180 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:47.801886 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:47.801890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:47.801892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:47.802011 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:47.802429 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:47.802432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:47.802434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:47.802694 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:47.802831 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:47.803522 (+ 691us) server_negotiation.cc:299] Negotiation successful
0806 19:42:47.803692 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":371,"threads_started":1}
I20260806 19:42:47.805025 29060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:47.785336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.67:40105)
0806 19:42:47.786248 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:47.786301 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:47.788203 (+ 1902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:47.789087 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:47.789096 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:47.789666 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:47.790287 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:47.790306 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:47.791585 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:47.791591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:47.793222 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:47.793245 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:47.794389 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:47.794397 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:47.794611 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:47.795437 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:47.795476 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:47.798281 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:47.801323 (+ 3042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:47.801331 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:47.801345 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:47.801771 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:47.802114 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:47.802118 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:47.802120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:47.802329 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:47.802821 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:47.802827 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:47.803162 (+ 335us) client_negotiation.cc:769] Sending connection context
0806 19:42:47.803402 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 19:42:47.803799 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":331,"threads_started":1}
I20260806 19:42:47.806546 29057 heartbeater.cc:347] Connected to a master server at 127.21.202.126:45677
I20260806 19:42:47.806989 29057 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:47.808003 29057 heartbeater.cc:511] Master 127.21.202.126:45677 requested a full tablet report, sending...
I20260806 19:42:47.809696 28594 ts_manager.cc:194] Registered new tserver with Master: 0a5049e41f514390804e74e692bc90bf (127.21.202.67:42981)
I20260806 19:42:47.810599 28594 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:40105
I20260806 19:42:47.821976 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.202.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.202.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.202.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.125.keytab.
Entry for principal oryx/127.21.202.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.125.keytab.
I20260806 19:42:47.881884 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.125:44803
--webserver_interface=127.21.202.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.125.keytab
--principal=oryx/127.21.202.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:35153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:45677,127.21.202.125:44803
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:48.233289 29063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:48.233618 29063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:48.233853 29063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:48.241576 29063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:48.241693 29063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:48.241856 29063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:48.241976 29063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:48.242074 29063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:42:48.252458 29063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:35153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:45677,127.21.202.125:44803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.125.keytab
--principal=oryx/127.21.202.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.125:44803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.202.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
W20260806 19:42:48.252972 29063 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.21.202.126:45677,127.21.202.125:44803'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 19:42:48.254139 29063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:48.255555 29063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:48.269554 29068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:48.269516 29071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:48.270007 28921 heartbeater.cc:503] Master 127.21.202.126:45677 was elected leader, sending a full tablet report...
W20260806 19:42:48.269582 29069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:48.270800 29063 server_base.cc:1034] running on GCE node
I20260806 19:42:48.271937 29063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:48.273751 29063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:48.275019 29063 hybrid_clock.cc:648] HybridClock initialized: now 1786045368274968 us; error 28 us; skew 500 ppm
I20260806 19:42:48.275384 29063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:48 dist-test-slave-77v9 krb5kdc[28549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045368, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:48.278942 29063 init.cc:377] Logged in from keytab as oryx/127.21.202.125@KRBTEST.COM (short username oryx)
I20260806 19:42:48.280694 29063 webserver.cc:460] Webserver started at http://127.21.202.125:40855/ using document root <none> and password file <none>
I20260806 19:42:48.281471 29063 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:48.281567 29063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:48.281837 29063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:48.285037 29063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/data/instance:
uuid: "8ad649fbf617413fa2b842dc829219f9"
format_stamp: "Formatted at 2026-08-06 19:42:48 on dist-test-slave-77v9"
server_key: "2599f080cf3abd38342bc05abd60edb9"
server_key_iv: "7d4100921d99355f1d14ae5c432a365d"
server_key_version: "encryptionkey@0"
I20260806 19:42:48.285882 29063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/wal/instance:
uuid: "8ad649fbf617413fa2b842dc829219f9"
format_stamp: "Formatted at 2026-08-06 19:42:48 on dist-test-slave-77v9"
server_key: "2599f080cf3abd38342bc05abd60edb9"
server_key_iv: "7d4100921d99355f1d14ae5c432a365d"
server_key_version: "encryptionkey@0"
I20260806 19:42:48.291363 29063 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 19:42:48.295130 29078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:48.296671 29063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 19:42:48.296921 29063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/wal
uuid: "8ad649fbf617413fa2b842dc829219f9"
format_stamp: "Formatted at 2026-08-06 19:42:48 on dist-test-slave-77v9"
server_key: "2599f080cf3abd38342bc05abd60edb9"
server_key_iv: "7d4100921d99355f1d14ae5c432a365d"
server_key_version: "encryptionkey@0"
I20260806 19:42:48.297173 29063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:48.316227 29063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:48.320796 29063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:48.321130 29063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:48.341395 29063 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.125:44803
I20260806 19:42:48.341423 29130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.125:44803 every 8 connection(s)
I20260806 19:42:48.342756 29063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/data/info.pb
I20260806 19:42:48.346800 29131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:48.347088 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 29063
I20260806 19:42:48.356056 29131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:42:48 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.202.125@KRBTEST.COM
Aug 06 19:42:48 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045368, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.125@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
I20260806 19:42:48.373525 29134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.350222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38984 (local address 127.21.202.125:44803)
0806 19:42:48.351042 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:48.351054 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:48.351095 (+ 41us) server_negotiation.cc:407] Connection header received
0806 19:42:48.353503 (+ 2408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:48.353524 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:48.353898 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:48.354309 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:48.355838 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.357275 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.358774 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.360070 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.360248 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.365325 (+ 5077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:48.365333 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:48.365343 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:48.365432 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:48.368764 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.369672 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.369678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.369685 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.369860 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.370376 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.370380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.370382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.371002 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:48.371215 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:48.371724 (+ 509us) server_negotiation.cc:299] Negotiation successful
0806 19:42:48.372121 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":362,"threads_started":1}
I20260806 19:42:48.375625 29135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.358712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51264 (local address 127.21.202.126:45677)
0806 19:42:48.359274 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:48.359281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:48.360055 (+ 774us) server_negotiation.cc:407] Connection header received
0806 19:42:48.360223 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:48.360232 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:48.360315 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:48.360454 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:48.361722 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.362998 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.364710 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.365206 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.365353 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.369099 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:48.369106 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:48.369111 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:48.369158 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:48.372370 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.373349 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.373354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.373356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.373498 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.373979 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.373984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.373989 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.374328 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:48.374503 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:48.375015 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 19:42:48.375244 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":291,"threads_started":1}
I20260806 19:42:48.375726 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.358710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:51264)
0806 19:42:48.359884 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:48.359930 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:48.360097 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:48.360510 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:48.360514 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:48.361291 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:48.361543 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.361553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.363150 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.363154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.364572 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.364584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.365425 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.365434 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.365625 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.366294 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:48.366321 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:48.368921 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:48.372495 (+ 3574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:48.372505 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:48.372512 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:48.373145 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:48.373663 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:48.373668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:48.373671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:48.373835 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:48.374474 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:48.374481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:48.374649 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 19:42:48.374926 (+ 277us) client_negotiation.cc:240] Negotiation successful
0806 19:42:48.375233 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"thread_start_us":771,"threads_started":1}
I20260806 19:42:48.378156 29131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:42:48 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045368, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.125@KRBTEST.COM for oryx/127.21.202.125@KRBTEST.COM
I20260806 19:42:48.391884 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.379320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:44803 (local address 127.0.0.1:38998)
0806 19:42:48.379524 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:48.379546 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:48.379693 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:48.380048 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:48.380051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:48.380399 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:48.380546 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.380554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.381612 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.381616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.382570 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.382581 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.383271 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.383276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.383403 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.383974 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:48.383991 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:48.386334 (+ 2343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:48.389661 (+ 3327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:48.389664 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:48.389667 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:48.390073 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:48.390434 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:48.390437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:48.390440 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:48.390564 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:48.391208 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:48.391211 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:48.391326 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 19:42:48.391438 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 19:42:48.391646 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 19:42:48.392237 29134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.379292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38998 (local address 127.21.202.125:44803)
0806 19:42:48.379462 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:48.379466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:48.379621 (+ 155us) server_negotiation.cc:407] Connection header received
0806 19:42:48.379813 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:48.379816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:48.379889 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:48.380025 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:48.380678 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.381474 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.382719 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.383116 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.383216 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.386526 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:48.386531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:48.386534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:48.386562 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:48.389512 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.390199 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.390204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.390206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.390329 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.390743 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.390748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.390750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.391096 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:48.391235 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:48.391550 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 19:42:48.391803 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":59}
I20260806 19:42:48.397261 29131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: Bootstrap starting.
I20260806 19:42:48.400367 29131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:48.401499 29131 log.cc:824] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:48.404431 29131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: No bootstrap required, opened a new log
I20260806 19:42:48.411636 29131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } }
I20260806 19:42:48.411971 29131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:48.412103 29131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ad649fbf617413fa2b842dc829219f9, State: Initialized, Role: FOLLOWER
I20260806 19:42:48.412699 29131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } }
I20260806 19:42:48.413729 29138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } } }
I20260806 19:42:48.414110 29138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:42:48.414839 29131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 19:42:48.423951 29149 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:42:48.424098 29149 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:42:48.427481 29063 master_runner.cc:428] Detected that this master 8ad649fbf617413fa2b842dc829219f9 is joining an existing cluster
I20260806 19:42:48.427605 29063 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.202.125:44803
I20260806 19:42:48.457767 29155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.435025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:44803 (local address 127.0.0.1:39014)
0806 19:42:48.438501 (+ 3476us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:48.438530 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:48.438721 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:48.439950 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:48.439957 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:48.441493 (+ 1536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:48.441932 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.441949 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.443605 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.443614 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.445740 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.445756 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.447195 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.447204 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.447471 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.448144 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:48.448172 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:48.449622 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:48.454034 (+ 4412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:48.454042 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:48.454049 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:48.454685 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:48.455174 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:48.455181 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:48.455187 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:48.455374 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:48.456277 (+ 903us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:48.456294 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:48.456524 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 19:42:48.456705 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 19:42:48.457114 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3312,"mutex_wait_us":107,"thread_start_us":3142,"threads_started":1}
I20260806 19:42:48.457760 29135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.435790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.21.202.126:45677)
0806 19:42:48.435980 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:48.435988 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:48.439123 (+ 3135us) server_negotiation.cc:407] Connection header received
0806 19:42:48.439320 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:48.439325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:48.439407 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:48.439530 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:48.442220 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.443355 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.445867 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.446577 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.446687 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.449667 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:48.449674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:48.449680 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:48.449738 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:48.453578 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.454486 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.454492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.454496 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.454663 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.455212 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.455217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.455221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.455780 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:48.456106 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:48.456864 (+ 758us) server_negotiation.cc:299] Negotiation successful
0806 19:42:48.457120 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 19:42:48.459312 29134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.435450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39014 (local address 127.21.202.125:44803)
0806 19:42:48.435652 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:48.435660 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:48.438630 (+ 2970us) server_negotiation.cc:407] Connection header received
0806 19:42:48.438897 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:48.438902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:48.438984 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:48.439103 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:48.442112 (+ 3009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.443392 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.445904 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.446766 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.446904 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.449814 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:48.449822 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:48.449830 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:48.449870 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:48.453891 (+ 4021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.454829 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.454836 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.454843 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.455038 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.455506 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.455512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.455517 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.455930 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:48.456206 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:48.458579 (+ 2373us) server_negotiation.cc:299] Negotiation successful
0806 19:42:48.458830 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 19:42:48.458993 29156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.436209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:51268)
0806 19:42:48.439017 (+ 2808us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:48.439041 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:48.439171 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:48.439551 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:48.439556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:48.441398 (+ 1842us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:48.442011 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.442026 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.443543 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.443549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.445681 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.445702 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.447037 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.447045 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.447286 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.448006 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:48.448031 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:48.449401 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:48.453756 (+ 4355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:48.453762 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:48.453766 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:48.454332 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:48.454828 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:48.454833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:48.454836 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:48.455064 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:48.456208 (+ 1144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:48.456213 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:48.456511 (+ 298us) client_negotiation.cc:769] Sending connection context
0806 19:42:48.458107 (+ 1596us) client_negotiation.cc:240] Negotiation successful
0806 19:42:48.458485 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2662,"mutex_wait_us":174,"thread_start_us":2620,"threads_started":1}
WARNING: no policy specified for oryx/127.21.202.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.202.124@KRBTEST.COM" created.
Aug 06 19:42:48 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045365, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.126@KRBTEST.COM for oryx/127.21.202.125@KRBTEST.COM
I20260806 19:42:48.492996 29158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.467532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:44803 (local address 127.0.0.1:39018)
0806 19:42:48.473487 (+ 5955us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:48.473540 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:48.473842 (+ 302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:48.474481 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:48.474491 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:48.475469 (+ 978us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:48.475804 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.475820 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.477571 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.477577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.479259 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.479277 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.480408 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.480417 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.480722 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.481576 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:48.481606 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:48.484889 (+ 3283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:48.489681 (+ 4792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:48.489697 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:48.489706 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:48.490356 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:48.490810 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:48.490815 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:48.490819 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:48.491027 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:48.491651 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:48.491660 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:48.491894 (+ 234us) client_negotiation.cc:769] Sending connection context
0806 19:42:48.492129 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 19:42:48.492482 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5684,"thread_start_us":5500,"threads_started":1}
I20260806 19:42:48.493033 29134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.467543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39018 (local address 127.21.202.125:44803)
0806 19:42:48.467742 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:48.467752 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:48.473649 (+ 5897us) server_negotiation.cc:407] Connection header received
0806 19:42:48.474140 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:48.474148 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:48.474267 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:48.474381 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:48.476032 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.477398 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.479452 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.480240 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.480382 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.485118 (+ 4736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:48.485127 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:48.485135 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:48.485199 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:48.489501 (+ 4302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.490514 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.490520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.490524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.490670 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.491203 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.491209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.491216 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.491511 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:48.491624 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:48.492235 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 19:42:48.492478 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 19:42:48.495663 28594 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.202.125:44803: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8ad649fbf617413fa2b842dc829219f9"
member_type: NON_VOTER
last_known_addr {
host: "127.21.202.125"
port: 44803
}
attrs {
promote: true
}
}
dest_uuid: "ec97e781650a49da8405e8eff3b5242e"
cas_config_opid_index: -1
I20260806 19:42:48.497288 28594 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.202.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.124.keytab.
Entry for principal oryx/127.21.202.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.124.keytab.
I20260806 19:42:48.501120 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.124:46083
--webserver_interface=127.21.202.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.124.keytab
--principal=oryx/127.21.202.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:35153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:45677,127.21.202.125:44803,127.21.202.124:46083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:42:48.509291 29105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:48.513737 29105 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 FOLLOWER]: Refusing update from remote peer ec97e781650a49da8405e8eff3b5242e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 19:42:48.513695 29161 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8ad649fbf617413fa2b842dc829219f9 (127.21.202.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } } }
I20260806 19:42:48.515193 29162 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:48.528574 29161 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.202.125:44803
I20260806 19:42:48.529681 29063 master.cc:537] Master@127.21.202.125:44803 shutting down...
I20260806 19:42:48.540879 29162 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } } }
I20260806 19:42:48.541198 29162 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:48.553886 29105 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8ad649fbf617413fa2b842dc829219f9 (127.21.202.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } } }
I20260806 19:42:48.603410 29171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } } }
I20260806 19:42:48.604002 29171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: This master's current role is: LEARNER
I20260806 19:42:48.603860 29138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec97e781650a49da8405e8eff3b5242e. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } } }
I20260806 19:42:48.604187 29138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: This master's current role is: LEARNER
I20260806 19:42:48.604399 29166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } } }
I20260806 19:42:48.604797 29166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: This master's current role is: LEARNER
I20260806 19:42:48.611528 29063 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 LEARNER]: Raft consensus shutting down.
I20260806 19:42:48.612129 29063 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 LEARNER]: Raft consensus is shut down!
I20260806 19:42:48.612311 29063 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: stopping tablet replica
I20260806 19:42:48.631858 29063 master.cc:552] Master@127.21.202.125:44803 shutdown complete.
I20260806 19:42:48.632028 29063 master_runner.cc:305] Clearing existing system tablet
I20260806 19:42:48.632712 29063 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:48.632942 29063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:48.636193 29063 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 19:42:48.641763 29173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:48.642181 29063 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:48.642488 29063 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260806 19:42:48.642572 29063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/wal
uuid: "8ad649fbf617413fa2b842dc829219f9"
format_stamp: "Formatted at 2026-08-06 19:42:48 on dist-test-slave-77v9"
server_key: "2599f080cf3abd38342bc05abd60edb9"
server_key_iv: "7d4100921d99355f1d14ae5c432a365d"
server_key_version: "encryptionkey@0"
I20260806 19:42:48.647770 29063 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:48.655189 29063 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 19:42:48.655390 29063 log.cc:1196] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 19:42:48.655930 29063 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: Deleting consensus metadata
I20260806 19:42:48.656201 29063 master_runner.cc:315] Copying system tablet from 127.21.202.126:45677
I20260806 19:42:48.663025 29063 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: tablet copy: Beginning tablet copy session from remote peer at address 127.21.202.126:45677
I20260806 19:42:48.686403 29135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.664278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51270 (local address 127.21.202.126:45677)
0806 19:42:48.664533 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:48.664544 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:48.665078 (+ 534us) server_negotiation.cc:407] Connection header received
0806 19:42:48.665453 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:48.665461 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:48.665552 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:48.665665 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:48.666975 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.668118 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.670254 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.670975 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.671106 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.679171 (+ 8065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:48.679179 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:48.679184 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:48.679230 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:48.683079 (+ 3849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.683986 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.683992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.683996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.684161 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:48.684645 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:48.684651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:48.684655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:48.685053 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:48.685214 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:48.685766 (+ 552us) server_negotiation.cc:299] Negotiation successful
0806 19:42:48.686007 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 19:42:48.686532 29179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.664184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:51270)
0806 19:42:48.664955 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:48.664980 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:48.665210 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:48.665719 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:48.665724 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:48.666378 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:48.666775 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.666794 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.668295 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.668303 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.670082 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.670099 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.671154 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.671161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.671407 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.677290 (+ 5883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:48.677330 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:48.678891 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:48.683245 (+ 4354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:48.683251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:48.683255 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:48.683824 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:48.684290 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:48.684296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:48.684299 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:48.684497 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:48.685199 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:48.685205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:48.685441 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 19:42:48.685629 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 19:42:48.686003 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":390,"threads_started":1}
I20260806 19:42:48.688004 28614 tablet_copy_service.cc:140] P ec97e781650a49da8405e8eff3b5242e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8ad649fbf617413fa2b842dc829219f9 ({username='oryx', principal='oryx/127.21.202.125@KRBTEST.COM'} at 127.0.0.1:51270)
I20260806 19:42:48.688524 28614 tablet_copy_service.cc:161] P ec97e781650a49da8405e8eff3b5242e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8ad649fbf617413fa2b842dc829219f9 at {username='oryx', principal='oryx/127.21.202.125@KRBTEST.COM'} at 127.0.0.1:51270: session id = 8ad649fbf617413fa2b842dc829219f9-00000000000000000000000000000000
I20260806 19:42:48.693471 28614 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 19:42:48.696421 29063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 19:42:48.704977 29063 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: tablet copy: Starting download of 0 data blocks...
I20260806 19:42:48.705542 29063 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: tablet copy: Starting download of 1 WAL segments...
I20260806 19:42:48.709718 29063 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 19:42:48.714664 28614 tablet_copy_service.cc:342] P ec97e781650a49da8405e8eff3b5242e: Request end of tablet copy session 8ad649fbf617413fa2b842dc829219f9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.202.125@KRBTEST.COM'} at 127.0.0.1:51270
I20260806 19:42:48.715015 28614 tablet_copy_service.cc:433] P ec97e781650a49da8405e8eff3b5242e: ending tablet copy session 8ad649fbf617413fa2b842dc829219f9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8ad649fbf617413fa2b842dc829219f9
I20260806 19:42:48.729396 29063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:48.732576 29180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:48.738787 29181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:48.742873 29183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:48.746577 29063 server_base.cc:1034] running on GCE node
I20260806 19:42:48.746973 29063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:48.747707 29063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:48.748948 29063 hybrid_clock.cc:648] HybridClock initialized: now 1786045368748877 us; error 35 us; skew 500 ppm
I20260806 19:42:48.749212 29063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:48 dist-test-slave-77v9 krb5kdc[28549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045368, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:48.751641 29063 init.cc:377] Logged in from keytab as oryx/127.21.202.125@KRBTEST.COM (short username oryx)
I20260806 19:42:48.759063 29063 webserver.cc:460] Webserver started at http://127.21.202.125:37943/ using document root <none> and password file <none>
I20260806 19:42:48.759634 29063 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:48.759750 29063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:48.763797 29063 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:42:48.766067 29190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:48.766626 29063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 19:42:48.766816 29063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/wal
uuid: "8ad649fbf617413fa2b842dc829219f9"
format_stamp: "Formatted at 2026-08-06 19:42:48 on dist-test-slave-77v9"
server_key: "2599f080cf3abd38342bc05abd60edb9"
server_key_iv: "7d4100921d99355f1d14ae5c432a365d"
server_key_version: "encryptionkey@0"
I20260806 19:42:48.767076 29063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:48.785861 29063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:48.790102 29063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:48.813114 29057 heartbeater.cc:503] Master 127.21.202.126:45677 was elected leader, sending a full tablet report...
I20260806 19:42:48.833150 29063 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.125:44803
I20260806 19:42:48.833175 29241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.125:44803 every 8 connection(s)
I20260806 19:42:48.834239 29063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-1/data/info.pb
I20260806 19:42:48.842012 29242 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:42:48.844656 29242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: Bootstrap starting.
I20260806 19:42:48.856350 29242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:42:48.856959 29242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: Bootstrap complete.
I20260806 19:42:48.857910 29242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } }
I20260806 19:42:48.858223 29242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8ad649fbf617413fa2b842dc829219f9, State: Initialized, Role: LEARNER
I20260806 19:42:48.858520 29242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } }
I20260806 19:42:48.859479 29245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } } }
I20260806 19:42:48.859882 29242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:48.859910 29245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: This master's current role is: LEARNER
I20260806 19:42:48.874179 29256 catalog_manager.cc:1259] Loaded cluster ID: 01055745776b4fecaad51f018b19e9d3
I20260806 19:42:48.874315 29256 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: loading cluster ID for follower catalog manager: success
I20260806 19:42:48.877162 29256 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: acquiring CA information for follower catalog manager: success
I20260806 19:42:48.879225 29256 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:42:48.932476 29162 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } } }
I20260806 19:42:48.932883 29162 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:48.939980 29158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.933380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:44803 (local address 127.0.0.1:41386)
0806 19:42:48.933584 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:48.933610 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:48.933771 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:48.934647 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:48.934652 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:48.934685 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:48.934898 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.934908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.936230 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.936237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.937827 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:48.937841 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.939102 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.939110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:48.939331 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.939394 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 19:42:48.939516 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 19:42:48.939664 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 19:42:48.940048 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:48.933509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41386 (local address 127.21.202.125:44803)
0806 19:42:48.934289 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:48.934294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:48.934312 (+ 18us) server_negotiation.cc:407] Connection header received
0806 19:42:48.934382 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:48.934386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:48.934454 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:48.934611 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:48.935052 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.936087 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.937990 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:48.938907 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:48.939128 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:48.939343 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:48.939548 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 19:42:48.939687 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":449,"threads_started":1}
I20260806 19:42:48.941356 29245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec97e781650a49da8405e8eff3b5242e. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } } }
I20260806 19:42:48.941643 29245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: This master's current role is: LEARNER
I20260806 19:42:48.942471 29257 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e: attempting to promote NON_VOTER 8ad649fbf617413fa2b842dc829219f9 to VOTER
I20260806 19:42:48.943259 29257 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } }
I20260806 19:42:48.945226 29217 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 LEARNER]: Refusing update from remote peer ec97e781650a49da8405e8eff3b5242e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 19:42:48.945812 29161 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 19:42:48.948885 29257 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8ad649fbf617413fa2b842dc829219f9 (127.21.202.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } }
I20260806 19:42:48.949599 29217 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8ad649fbf617413fa2b842dc829219f9 (127.21.202.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } }
I20260806 19:42:48.953105 29162 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } }
I20260806 19:42:48.953374 29162 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:48.953761 29162 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } }
I20260806 19:42:48.954016 29162 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:48.953933 29245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } }
I20260806 19:42:48.954238 29245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 19:42:49.002408 29163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:49.002801 29163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:49.002950 29163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:49.014834 29163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:49.015022 29163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:49.015126 29163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:49.015208 29163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:49.015268 29163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:42:49.030827 29163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:35153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:45677,127.21.202.125:44803,127.21.202.124:46083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.124.keytab
--principal=oryx/127.21.202.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.124:46083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.202.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:49.032611 29163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:49.034149 29163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:49.048982 29265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:49.049031 29266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:49.049633 29268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:49.050257 29163 server_base.cc:1034] running on GCE node
I20260806 19:42:49.051064 29163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:49.052559 29163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:49.053778 29163 hybrid_clock.cc:648] HybridClock initialized: now 1786045369053674 us; error 92 us; skew 500 ppm
I20260806 19:42:49.054121 29163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:49 dist-test-slave-77v9 krb5kdc[28549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045369, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:49.057518 29163 init.cc:377] Logged in from keytab as oryx/127.21.202.124@KRBTEST.COM (short username oryx)
I20260806 19:42:49.059266 29163 webserver.cc:460] Webserver started at http://127.21.202.124:45965/ using document root <none> and password file <none>
I20260806 19:42:49.059993 29163 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:49.060142 29163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:49.060465 29163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:49.063519 29163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/data/instance:
uuid: "cfedfa797c6b45538571329bd5c9b53c"
format_stamp: "Formatted at 2026-08-06 19:42:49 on dist-test-slave-77v9"
server_key: "dd50dacaa1fedfdb9cd94beeaeaf1448"
server_key_iv: "691199450ee4ee419eead331d936afb8"
server_key_version: "encryptionkey@0"
I20260806 19:42:49.064350 29163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/wal/instance:
uuid: "cfedfa797c6b45538571329bd5c9b53c"
format_stamp: "Formatted at 2026-08-06 19:42:49 on dist-test-slave-77v9"
server_key: "dd50dacaa1fedfdb9cd94beeaeaf1448"
server_key_iv: "691199450ee4ee419eead331d936afb8"
server_key_version: "encryptionkey@0"
I20260806 19:42:49.070065 29163 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 19:42:49.073596 29275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:49.075124 29163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:42:49.075341 29163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/wal
uuid: "cfedfa797c6b45538571329bd5c9b53c"
format_stamp: "Formatted at 2026-08-06 19:42:49 on dist-test-slave-77v9"
server_key: "dd50dacaa1fedfdb9cd94beeaeaf1448"
server_key_iv: "691199450ee4ee419eead331d936afb8"
server_key_version: "encryptionkey@0"
I20260806 19:42:49.075584 29163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:49.105527 29163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:49.110618 29163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:49.110968 29163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:49.133285 29163 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.124:46083
I20260806 19:42:49.133297 29326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.124:46083 every 8 connection(s)
I20260806 19:42:49.134806 29163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/data/info.pb
I20260806 19:42:49.138972 29327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:49.140893 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 29163
I20260806 19:42:49.149029 29327 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:42:49 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.202.124@KRBTEST.COM
Aug 06 19:42:49 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045369, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.124@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
I20260806 19:42:49.166841 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.143486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55532 (local address 127.21.202.124:46083)
0806 19:42:49.144304 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:49.144318 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:49.144361 (+ 43us) server_negotiation.cc:407] Connection header received
0806 19:42:49.146173 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:49.146196 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:49.146636 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:49.147022 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:49.148503 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.149956 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.151245 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.152179 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.152410 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.157638 (+ 5228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:49.157649 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:49.157660 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:49.157720 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:49.161562 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.162403 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.162410 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.162417 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.162614 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.163285 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.163293 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.163297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.164108 (+ 811us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:49.164386 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:49.165065 (+ 679us) server_negotiation.cc:299] Negotiation successful
0806 19:42:49.165575 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":376,"threads_started":1}
I20260806 19:42:49.169126 29135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.151837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45358 (local address 127.21.202.126:45677)
0806 19:42:49.152046 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:49.152053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:49.152924 (+ 871us) server_negotiation.cc:407] Connection header received
0806 19:42:49.153091 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:49.153099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:49.153189 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:49.153281 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:49.154974 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.155761 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.157395 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.157968 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.158103 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.162368 (+ 4265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:49.162379 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:49.162387 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:49.162439 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:49.166057 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.166930 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.166936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.166939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.167103 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.167533 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.167538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.167542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.167829 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:49.167989 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:49.168528 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 19:42:49.168733 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 19:42:49.169214 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.151746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:45358)
0806 19:42:49.152676 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:49.152730 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:49.152937 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:49.153388 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:49.153395 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:49.154342 (+ 947us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:49.154767 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.154781 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.155940 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.155949 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.157237 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.157251 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.158182 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.158189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.158397 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.159052 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:49.159108 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:49.162106 (+ 2998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:49.166187 (+ 4081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.166200 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.166207 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.166801 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.167205 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.167210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.167213 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.167410 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.167964 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:49.167971 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:49.168153 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 19:42:49.168345 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 19:42:49.168703 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":506,"threads_started":1}
I20260806 19:42:49.170755 29327 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:42:49 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045369, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.124@KRBTEST.COM for oryx/127.21.202.125@KRBTEST.COM
I20260806 19:42:49.184418 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.171666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:44803 (local address 127.0.0.1:41396)
0806 19:42:49.171915 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:49.171934 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:49.172044 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:49.172377 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:49.172381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:49.172728 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:49.172940 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.172950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.174089 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.174094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.175133 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.175145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.175846 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.175851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.175974 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.176582 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:49.176600 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:49.178687 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:49.182273 (+ 3586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.182277 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.182279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.182662 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.183055 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.183062 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.183066 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.183214 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.183755 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:49.183758 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:49.183868 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 19:42:49.183977 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 19:42:49.184175 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 19:42:49.184671 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.171723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41396 (local address 127.21.202.125:44803)
0806 19:42:49.171954 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:49.171962 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:49.171983 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:42:49.172177 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:49.172181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:49.172246 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:49.172356 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:49.173078 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.173908 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.175292 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.175699 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.175801 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.178912 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:49.178918 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:49.178921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:49.178955 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:49.182144 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.182764 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.182768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.182770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.182941 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.183348 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.183352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.183354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.183647 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:49.183774 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:49.184104 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 19:42:49.184299 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 19:42:49.185721 29327 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:42:49 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045369, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.124@KRBTEST.COM for oryx/127.21.202.124@KRBTEST.COM
I20260806 19:42:49.198731 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.186590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:46083 (local address 127.0.0.1:55536)
0806 19:42:49.186749 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:49.186765 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:49.186887 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:49.187143 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:49.187147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:49.187485 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:49.187637 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.187645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.188547 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.188551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.189563 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.189585 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.190263 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.190268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.190440 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.191046 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:49.191064 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:49.193079 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:49.196284 (+ 3205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.196289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.196292 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.196680 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.197122 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.197126 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.197128 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.197306 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.197840 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:49.197844 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:49.198011 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 19:42:49.198140 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 19:42:49.198355 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 19:42:49.198838 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.186566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55536 (local address 127.21.202.124:46083)
0806 19:42:49.186733 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:49.186739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:49.186854 (+ 115us) server_negotiation.cc:407] Connection header received
0806 19:42:49.186975 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:49.186978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:49.187031 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:49.187105 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:49.187750 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.188414 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.189700 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.190110 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.190203 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.193305 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:49.193311 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:49.193314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:49.193345 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:49.196122 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.196850 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.196860 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.196867 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.197002 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.197410 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.197414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.197416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.197715 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:49.197836 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:49.198250 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 19:42:49.198518 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":56}
I20260806 19:42:49.203992 29327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: Bootstrap starting.
I20260806 19:42:49.207546 29327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:49.208634 29327 log.cc:824] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:49.211325 29327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: No bootstrap required, opened a new log
I20260806 19:42:49.219404 29327 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } }
I20260806 19:42:49.219726 29327 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:49.219890 29327 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfedfa797c6b45538571329bd5c9b53c, State: Initialized, Role: FOLLOWER
I20260806 19:42:49.220603 29327 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } }
I20260806 19:42:49.221763 29333 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } } }
I20260806 19:42:49.222095 29333 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:42:49.222905 29327 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: configured and running, proceeding with master startup.
W20260806 19:42:49.232282 29344 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:42:49.232426 29344 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:42:49.237279 29163 master_runner.cc:428] Detected that this master cfedfa797c6b45538571329bd5c9b53c is joining an existing cluster
I20260806 19:42:49.237443 29163 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.202.124:46083
I20260806 19:42:49.266893 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.244902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55550 (local address 127.21.202.124:46083)
0806 19:42:49.245183 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:49.245198 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:49.248537 (+ 3339us) server_negotiation.cc:407] Connection header received
0806 19:42:49.248796 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:49.248802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:49.248920 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:49.249033 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:49.250693 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.252092 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.254465 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.255367 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.255470 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.258478 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:49.258485 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:49.258491 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:49.258530 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:49.263031 (+ 4501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.263864 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.263870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.263874 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.264042 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.264507 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.264514 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.264517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.265035 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:49.265207 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:49.265845 (+ 638us) server_negotiation.cc:299] Negotiation successful
0806 19:42:49.266437 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 19:42:49.268079 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.244520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:46083 (local address 127.0.0.1:55550)
0806 19:42:49.248373 (+ 3853us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:49.248402 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:49.248622 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:49.249215 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:49.249220 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:49.250076 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:49.250526 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.250541 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.252355 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.252365 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.254289 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.254305 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.255843 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.255852 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.256109 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.256972 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:49.257019 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:49.258267 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:49.263185 (+ 4918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.263190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.263195 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.263724 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.264196 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.264204 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.264210 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.264382 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.265279 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:49.265284 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:49.265441 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 19:42:49.267190 (+ 1749us) client_negotiation.cc:240] Negotiation successful
0806 19:42:49.267547 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3634,"thread_start_us":3409,"threads_started":1}
I20260806 19:42:49.270536 29135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.244470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45362 (local address 127.21.202.126:45677)
0806 19:42:49.244673 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:49.244681 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:49.250155 (+ 5474us) server_negotiation.cc:407] Connection header received
0806 19:42:49.250818 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:49.250828 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:49.250946 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:49.251040 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:49.252985 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.254314 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.257171 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.257958 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.258068 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.262098 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:49.262106 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:49.262111 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:49.262157 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:49.266718 (+ 4561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.267796 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.267804 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.267810 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.267995 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.268522 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.268529 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.268533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.268976 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:49.269139 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:49.269888 (+ 749us) server_negotiation.cc:299] Negotiation successful
0806 19:42:49.270139 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 19:42:49.271050 29351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.245107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:45362)
0806 19:42:49.250021 (+ 4914us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:49.250045 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:49.250603 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:49.251340 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:49.251348 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:49.252290 (+ 942us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:49.252789 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.252801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.254529 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.254535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.256983 (+ 2448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.256998 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.258451 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.258457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.258660 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.259722 (+ 1062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:49.259801 (+ 79us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:49.261841 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:49.266990 (+ 5149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.266995 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.266998 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.267649 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.268162 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.268166 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.268170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.268373 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.269231 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:49.269247 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:49.269491 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 19:42:49.269649 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 19:42:49.270423 (+ 774us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4726,"thread_start_us":5065,"threads_started":1}
I20260806 19:42:49.273820 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.244446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.21.202.125:44803)
0806 19:42:49.244673 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:49.244681 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:49.252230 (+ 7549us) server_negotiation.cc:407] Connection header received
0806 19:42:49.252828 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:49.252835 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:49.252939 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:49.253019 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:49.254564 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.255809 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.258259 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.259085 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.259234 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.262717 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:49.262725 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:49.262732 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:49.262807 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:49.266906 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.270487 (+ 3581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.270493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.270496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.270920 (+ 424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.271467 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.271474 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.271477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.271891 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:49.272044 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:49.272681 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 19:42:49.273411 (+ 730us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 19:42:49.275048 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.245322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:44803 (local address 127.0.0.1:41410)
0806 19:42:49.252114 (+ 6792us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:49.252135 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:49.252570 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:49.253257 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:49.253266 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:49.253985 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:49.254374 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.254397 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.255999 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.256005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.258073 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.258094 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.259709 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.259718 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.260026 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.260889 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:49.260923 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:49.262486 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:49.267724 (+ 5238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.267731 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.267748 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.270322 (+ 2574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.271103 (+ 781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.271110 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.271114 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.271307 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.272135 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:49.272141 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:49.272362 (+ 221us) client_negotiation.cc:769] Sending connection context
0806 19:42:49.274100 (+ 1738us) client_negotiation.cc:240] Negotiation successful
0806 19:42:49.274480 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6658,"thread_start_us":6934,"threads_started":1}
I20260806 19:42:49.275454 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 28655
Aug 06 19:42:49 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045365, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.126@KRBTEST.COM for oryx/127.21.202.124@KRBTEST.COM
Aug 06 19:42:49 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.202.65@KRBTEST.COM
I20260806 19:42:49.297142 29158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.278160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:46083 (local address 127.0.0.1:55552)
0806 19:42:49.278708 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:49.278730 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:49.278843 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:49.279327 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:49.279333 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:49.279945 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:49.280206 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.280221 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.282476 (+ 2255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.282482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.284043 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.284056 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.285106 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.285112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.285305 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.285979 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:49.286023 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:49.289272 (+ 3249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:49.293836 (+ 4564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.293843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.293849 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.294427 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.294925 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.294931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.294935 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.295121 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.295865 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:49.295871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:49.296085 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 19:42:49.296270 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 19:42:49.296616 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421}
I20260806 19:42:49.297178 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.278703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55552 (local address 127.21.202.124:46083)
0806 19:42:49.278887 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:49.278893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:49.278913 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:42:49.279005 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:49.279010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:49.279139 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:49.280431 (+ 1292us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:49.280624 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.282315 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.284309 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.284943 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.285092 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.289508 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:49.289517 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:49.289523 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:49.289563 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:49.293668 (+ 4105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.294573 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.294580 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.294583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.294767 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.295266 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.295272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.295276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.295692 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:49.295849 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:49.296422 (+ 573us) server_negotiation.cc:299] Negotiation successful
0806 19:42:49.296704 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 19:42:49.299041 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.277188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43274 (local address 127.21.202.65:36007)
0806 19:42:49.277932 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:49.277942 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:49.277976 (+ 34us) server_negotiation.cc:407] Connection header received
0806 19:42:49.278065 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:49.278075 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:49.278390 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:49.280459 (+ 2069us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:49.280758 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.281950 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.283948 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.285043 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.285860 (+ 817us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.291120 (+ 5260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:49.291139 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:49.291162 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:49.291217 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:49.295270 (+ 4053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.296222 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.296263 (+ 41us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.296270 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.296445 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.297008 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.297014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.297018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.297516 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:49.297863 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:49.298235 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 19:42:49.298541 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":1010,"threads_started":1}
I20260806 19:42:49.299365 28593 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.202.124:46083: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c"
member_type: NON_VOTER
last_known_addr {
host: "127.21.202.124"
port: 46083
}
attrs {
promote: true
}
}
dest_uuid: "ec97e781650a49da8405e8eff3b5242e"
cas_config_opid_index: 6
I20260806 19:42:49.300375 28593 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: NON_VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: true } }
I20260806 19:42:49.302510 28781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:49.302935 29217 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 FOLLOWER]: Refusing update from remote peer ec97e781650a49da8405e8eff3b5242e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 19:42:49.303357 29302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:49.303795 29162 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 19:42:49.307673 29302 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 FOLLOWER]: Refusing update from remote peer ec97e781650a49da8405e8eff3b5242e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 19:42:49.307456 29162 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER cfedfa797c6b45538571329bd5c9b53c (127.21.202.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: NON_VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: true } } }
I20260806 19:42:49.312961 29161 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: NON_VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:49.314764 29217 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER cfedfa797c6b45538571329bd5c9b53c (127.21.202.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: NON_VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: true } } }
I20260806 19:42:49.319634 29245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: NON_VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: true } } }
I20260806 19:42:49.319983 29245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:42:49.320520 29162 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.202.124:46083
I20260806 19:42:49.321237 29163 master.cc:537] Master@127.21.202.124:46083 shutting down...
I20260806 19:42:49.322213 29257 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: NON_VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: true } } }
I20260806 19:42:49.322548 29257 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:49.322927 29162 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: NON_VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: true } } }
I20260806 19:42:49.323254 29162 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:49.334266 29302 raft_consensus.cc:858] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } }
W20260806 19:42:49.334476 29302 raft_consensus.cc:1565] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } }
I20260806 19:42:49.341638 29302 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8ad649fbf617413fa2b842dc829219f9 (127.21.202.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } } }
I20260806 19:42:49.361286 29362 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } } }
I20260806 19:42:49.361845 29362 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 19:42:49.362277 29358 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } } }
I20260806 19:42:49.362653 29358 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 19:42:49.364531 29162 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: NON_VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: true } } }
I20260806 19:42:49.364888 29162 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:49.365463 29333 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec97e781650a49da8405e8eff3b5242e. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: NON_VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: true } } }
I20260806 19:42:49.365680 29333 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: This master's current role is: NON_PARTICIPANT
W20260806 19:42:49.366122 28578 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 19:42:49.367275 29163 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260806 19:42:49.367843 29163 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260806 19:42:49.368067 29163 tablet_replica.cc:330] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: stopping tablet replica
W20260806 19:42:49.373183 28578 consensus_peers.cc:487] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e -> Peer cfedfa797c6b45538571329bd5c9b53c (127.21.202.124:46083): Couldn't send request to peer cfedfa797c6b45538571329bd5c9b53c. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260806 19:42:49.376600 29163 master.cc:552] Master@127.21.202.124:46083 shutdown complete.
I20260806 19:42:49.376691 29163 master_runner.cc:305] Clearing existing system tablet
I20260806 19:42:49.377074 29163 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:49.377209 29163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:49.379467 29163 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 19:42:49.381080 29365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:49.381458 29163 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:49.381711 29163 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260806 19:42:49.381826 29163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/wal
uuid: "cfedfa797c6b45538571329bd5c9b53c"
format_stamp: "Formatted at 2026-08-06 19:42:49 on dist-test-slave-77v9"
server_key: "dd50dacaa1fedfdb9cd94beeaeaf1448"
server_key_iv: "691199450ee4ee419eead331d936afb8"
server_key_version: "encryptionkey@0"
I20260806 19:42:49.385106 29163 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:49.389784 29163 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 19:42:49.389947 29163 log.cc:1196] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 19:42:49.390457 29163 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: Deleting consensus metadata
I20260806 19:42:49.390705 29163 master_runner.cc:315] Copying system tablet from 127.21.202.126:45677
I20260806 19:42:49.393237 29163 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: tablet copy: Beginning tablet copy session from remote peer at address 127.21.202.126:45677
I20260806 19:42:49.405884 29135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.394050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45364 (local address 127.21.202.126:45677)
0806 19:42:49.394285 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:49.394295 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:49.394828 (+ 533us) server_negotiation.cc:407] Connection header received
0806 19:42:49.395039 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:49.395044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:49.395126 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:49.395244 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:49.396034 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.396765 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.398064 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.398489 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.398628 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.400506 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:49.400512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:49.400517 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:49.400558 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:49.403272 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.403912 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.403919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.403922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.404052 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:49.404447 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:49.404454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:49.404457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:49.404744 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:49.404930 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:49.405309 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 19:42:49.405522 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:42:49.405992 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.394009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:45364)
0806 19:42:49.394710 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:49.394727 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:49.394894 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:49.395260 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:49.395265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:49.395677 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:49.395882 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.395890 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.396998 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.397004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.397915 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.397926 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.398639 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.398645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.398856 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.399521 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:49.399539 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:49.400310 (+ 771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:49.403416 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.403422 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.403429 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.403795 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.404170 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:49.404175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:49.404179 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:49.404332 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:49.404895 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:49.404899 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:49.405062 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 19:42:49.405205 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 19:42:49.405517 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":443,"threads_started":1}
I20260806 19:42:49.406891 28614 tablet_copy_service.cc:140] P ec97e781650a49da8405e8eff3b5242e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cfedfa797c6b45538571329bd5c9b53c ({username='oryx', principal='oryx/127.21.202.124@KRBTEST.COM'} at 127.0.0.1:45364)
I20260806 19:42:49.407068 28614 tablet_copy_service.cc:161] P ec97e781650a49da8405e8eff3b5242e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cfedfa797c6b45538571329bd5c9b53c at {username='oryx', principal='oryx/127.21.202.124@KRBTEST.COM'} at 127.0.0.1:45364: session id = cfedfa797c6b45538571329bd5c9b53c-00000000000000000000000000000000
I20260806 19:42:49.409461 28614 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 19:42:49.411031 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 19:42:49.415917 29163 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: tablet copy: Starting download of 0 data blocks...
I20260806 19:42:49.416226 29163 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: tablet copy: Starting download of 1 WAL segments...
I20260806 19:42:49.418371 29163 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 19:42:49.421525 28614 tablet_copy_service.cc:342] P ec97e781650a49da8405e8eff3b5242e: Request end of tablet copy session cfedfa797c6b45538571329bd5c9b53c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.202.124@KRBTEST.COM'} at 127.0.0.1:45364
I20260806 19:42:49.421660 28614 tablet_copy_service.cc:433] P ec97e781650a49da8405e8eff3b5242e: ending tablet copy session cfedfa797c6b45538571329bd5c9b53c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cfedfa797c6b45538571329bd5c9b53c
I20260806 19:42:49.443579 29163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:49.446868 29372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:49.447813 29373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:49.462069 29375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:49.462155 29163 server_base.cc:1034] running on GCE node
I20260806 19:42:49.462568 29163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:49.463332 29163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:49.464556 29163 hybrid_clock.cc:648] HybridClock initialized: now 1786045369464509 us; error 45 us; skew 500 ppm
I20260806 19:42:49.464870 29163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:49 dist-test-slave-77v9 krb5kdc[28549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045369, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:49.468542 29163 init.cc:377] Logged in from keytab as oryx/127.21.202.124@KRBTEST.COM (short username oryx)
I20260806 19:42:49.470074 29163 webserver.cc:460] Webserver started at http://127.21.202.124:40883/ using document root <none> and password file <none>
I20260806 19:42:49.470438 29163 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:49.470556 29163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:49.473932 29163 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 19:42:49.474534 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 28655
I20260806 19:42:49.481472 29382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:49.481961 29163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 19:42:49.482100 29163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/wal
uuid: "cfedfa797c6b45538571329bd5c9b53c"
format_stamp: "Formatted at 2026-08-06 19:42:49 on dist-test-slave-77v9"
server_key: "dd50dacaa1fedfdb9cd94beeaeaf1448"
server_key_iv: "691199450ee4ee419eead331d936afb8"
server_key_version: "encryptionkey@0"
I20260806 19:42:49.482309 29163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:49.494875 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:36007
--local_ip_for_outbound_sockets=127.21.202.65
--tserver_master_addrs=127.21.202.126:45677
--webserver_port=33463
--webserver_interface=127.21.202.65
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab
--principal=oryx/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:35153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.202.126:45677,127.21.202.125:44803
--tserver_master_addrs=127.21.202.126:45677,127.21.202.125:44803,127.21.202.124:46083 with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:42:49.521881 29163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:49.526443 29163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:49.549844 29163 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.124:46083
I20260806 19:42:49.549889 29434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.124:46083 every 8 connection(s)
I20260806 19:42:49.551024 29163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/master-2/data/info.pb
I20260806 19:42:49.557376 29435 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:42:49.561609 29435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: Bootstrap starting.
I20260806 19:42:49.577391 29435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:42:49.577986 29435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: Bootstrap complete.
I20260806 19:42:49.579203 29435 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: NON_VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: true } }
I20260806 19:42:49.579537 29435 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cfedfa797c6b45538571329bd5c9b53c, State: Initialized, Role: LEARNER
I20260806 19:42:49.579823 29435 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: NON_VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: true } }
I20260806 19:42:49.581100 29435 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:49.580793 29438 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: NON_VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: true } } }
I20260806 19:42:49.581444 29438 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: This master's current role is: LEARNER
I20260806 19:42:49.618216 29449 catalog_manager.cc:1259] Loaded cluster ID: 01055745776b4fecaad51f018b19e9d3
I20260806 19:42:49.618362 29449 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: loading cluster ID for follower catalog manager: success
I20260806 19:42:49.622047 29449 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: acquiring CA information for follower catalog manager: success
I20260806 19:42:49.624518 29449 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:42:49.767740 29161 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: NON_VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: true } } }
I20260806 19:42:49.768150 29161 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:49.774739 29450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.768260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55566 (local address 127.21.202.124:46083)
0806 19:42:49.769326 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:49.769331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:49.769348 (+ 17us) server_negotiation.cc:407] Connection header received
0806 19:42:49.769418 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:49.769425 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:49.769497 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:49.769636 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:49.770066 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.771041 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.772587 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.773474 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.773699 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.773913 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:49.774179 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 19:42:49.774368 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":938,"thread_start_us":791,"threads_started":1}
I20260806 19:42:49.774730 29158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:49.768103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:46083 (local address 127.0.0.1:55566)
0806 19:42:49.768398 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:49.768420 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:49.768552 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:49.769658 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:49.769663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:49.769691 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:49.769902 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.769911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.771184 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.771192 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.772416 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:49.772429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:49.773700 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:49.773706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:49.773965 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:49.774008 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 19:42:49.774153 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 19:42:49.774334 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260806 19:42:49.776007 29410 raft_consensus.cc:1203] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260806 19:42:49.776532 29438 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec97e781650a49da8405e8eff3b5242e. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: NON_VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: true } } }
I20260806 19:42:49.776948 29438 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: This master's current role is: LEARNER
I20260806 19:42:49.777269 29161 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e: attempting to promote NON_VOTER cfedfa797c6b45538571329bd5c9b53c to VOTER
I20260806 19:42:49.777882 29161 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } }
I20260806 19:42:49.779799 29410 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 LEARNER]: Refusing update from remote peer ec97e781650a49da8405e8eff3b5242e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 19:42:49.780002 29217 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 FOLLOWER]: Refusing update from remote peer ec97e781650a49da8405e8eff3b5242e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 19:42:49.780594 29161 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 19:42:49.781020 29257 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 19:42:49.786896 29257 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, cfedfa797c6b45538571329bd5c9b53c (127.21.202.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } } }
I20260806 19:42:49.788270 29410 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, cfedfa797c6b45538571329bd5c9b53c (127.21.202.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } } }
I20260806 19:42:49.791299 29438 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } } }
I20260806 19:42:49.791561 29438 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:42:49.791603 29355 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } } }
I20260806 19:42:49.792060 29355 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:49.792326 29217 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, cfedfa797c6b45538571329bd5c9b53c (127.21.202.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } } }
I20260806 19:42:49.792659 29355 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } } }
I20260806 19:42:49.792973 29355 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:49.793171 29257 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } } }
I20260806 19:42:49.793431 29257 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:49.795452 29245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec97e781650a49da8405e8eff3b5242e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } } }
I20260806 19:42:49.795743 29245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 19:42:49.885076 29387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:49.885362 29387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:49.885511 29387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:49.893245 29387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:49.893366 29387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:49.893544 29387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:42:49.904798 29387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:35153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab
--principal=oryx/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:36007
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=33463
--tserver_master_addrs=127.21.202.126:45677,127.21.202.125:44803,127.21.202.124:46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:49.906714 29387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:49.908181 29387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:49.923105 29456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:49.923244 29459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:49.923269 29457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:49.924250 29387 server_base.cc:1034] running on GCE node
I20260806 19:42:49.925024 29387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:49.926079 29387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:49.927320 29387 hybrid_clock.cc:648] HybridClock initialized: now 1786045369927239 us; error 63 us; skew 500 ppm
I20260806 19:42:49.927692 29387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:49 dist-test-slave-77v9 krb5kdc[28549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045369, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:49.931303 29387 init.cc:377] Logged in from keytab as oryx/127.21.202.65@KRBTEST.COM (short username oryx)
I20260806 19:42:49.933144 29387 webserver.cc:460] Webserver started at http://127.21.202.65:33463/ using document root <none> and password file <none>
I20260806 19:42:49.933845 29387 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:49.933980 29387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:49.940083 29387 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 19:42:49.943109 29466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:49.944721 29387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:42:49.944965 29387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "d64104c8ac3b4b4fb0f11c7d035d575c"
format_stamp: "Formatted at 2026-08-06 19:42:46 on dist-test-slave-77v9"
server_key: "0e3484d57efdd78b4378c49d9a95c761"
server_key_iv: "c1b46bd63db657b397f82dc9964128f2"
server_key_version: "encryptionkey@0"
I20260806 19:42:49.945963 29387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:49.962643 29387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:49.967428 29387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:49.967743 29387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:49.968976 29387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:49.970499 29387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:49.970620 29387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:49.970757 29387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:49.970855 29387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:50.009101 29387 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:36007
I20260806 19:42:50.009166 29580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:36007 every 8 connection(s)
I20260806 19:42:50.010599 29387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:42:50.011475 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 29387
I20260806 19:42:50.011858 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 28792
Aug 06 19:42:50 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045369, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.125@KRBTEST.COM
Aug 06 19:42:50 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045369, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
Aug 06 19:42:50 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045369, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.124@KRBTEST.COM
I20260806 19:42:50.048879 29589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.015371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:60923 (local address 127.21.202.125:44803)
0806 19:42:50.025517 (+ 10146us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:50.025526 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:50.025548 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:42:50.025615 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:50.025621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:50.025706 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:50.025833 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:50.027388 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.028594 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.030275 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.030998 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.031133 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.038888 (+ 7755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:50.038898 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:50.038905 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:50.038952 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:50.043684 (+ 4732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.044861 (+ 1177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.044868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.044871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.045040 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.045598 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.045604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.045610 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.046011 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:50.046152 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:50.048242 (+ 2090us) server_negotiation.cc:299] Negotiation successful
0806 19:42:50.048491 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9979,"thread_start_us":9814,"threads_started":1}
I20260806 19:42:50.049863 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.014544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:44803 (local address 127.21.202.65:60923)
0806 19:42:50.015579 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:50.015633 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:50.018797 (+ 3164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:50.025885 (+ 7088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:50.025892 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:50.026468 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:50.027123 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:50.027146 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.028789 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.028794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:50.030104 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:50.030116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.031473 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.031480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:50.031817 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.032841 (+ 1024us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:50.032875 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:50.038546 (+ 5671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:50.043828 (+ 5282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:50.043832 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:50.043835 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:50.044672 (+ 837us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:50.045186 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:50.045191 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:50.045195 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:50.045467 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:50.046241 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:50.046253 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:50.046778 (+ 525us) client_negotiation.cc:769] Sending connection context
0806 19:42:50.047111 (+ 333us) client_negotiation.cc:240] Negotiation successful
0806 19:42:50.047744 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":6,"spinlock_wait_cycles":7936,"thread_start_us":10083,"threads_started":1}
I20260806 19:42:50.050326 29450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.014276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:48307 (local address 127.21.202.124:46083)
0806 19:42:50.014507 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:50.014513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:50.023839 (+ 9326us) server_negotiation.cc:407] Connection header received
0806 19:42:50.024192 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:50.024201 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:50.024312 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:50.024406 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:50.027759 (+ 3353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.028863 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.032320 (+ 3457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.033061 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.033196 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.039704 (+ 6508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:50.039712 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:50.039716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:50.039766 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:50.043577 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.045514 (+ 1937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.045523 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.045527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.045700 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.046481 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.046486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.046489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.046829 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:50.046967 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:50.049407 (+ 2440us) server_negotiation.cc:299] Negotiation successful
0806 19:42:50.049667 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 19:42:50 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.202.66@KRBTEST.COM
I20260806 19:42:50.051592 29590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.015644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:53191 (local address 127.21.202.126:45677)
0806 19:42:50.023129 (+ 7485us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:50.023138 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:50.026771 (+ 3633us) server_negotiation.cc:407] Connection header received
0806 19:42:50.026866 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:50.026872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:50.026956 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:50.027082 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:50.029017 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.030238 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.032344 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.033105 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.033225 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.039621 (+ 6396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:50.039628 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:50.039632 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:50.039675 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:50.044426 (+ 4751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.045271 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.045276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.045280 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.045436 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.046276 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.046281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.046285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.046663 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:50.046797 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:50.050972 (+ 4175us) server_negotiation.cc:299] Negotiation successful
0806 19:42:50.051212 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7294,"thread_start_us":7141,"threads_started":1}
I20260806 19:42:50.051759 29583 heartbeater.cc:347] Connected to a master server at 127.21.202.125:44803
I20260806 19:42:50.052068 29591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.014596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:46083 (local address 127.21.202.65:48307)
0806 19:42:50.023663 (+ 9067us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:50.023686 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:50.024005 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:50.025051 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:50.025067 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:50.026198 (+ 1131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:50.027555 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:50.027568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.030458 (+ 2890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.030463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:50.032144 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:50.032159 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.034057 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.034065 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:50.034248 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.036447 (+ 2199us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:50.036476 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:50.038996 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:50.043774 (+ 4778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:50.043783 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:50.043805 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:50.045367 (+ 1562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:50.046152 (+ 785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:50.046157 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:50.046161 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:50.046338 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:50.047453 (+ 1115us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:50.047459 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:50.047587 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 19:42:50.051110 (+ 3523us) client_negotiation.cc:240] Negotiation successful
0806 19:42:50.051401 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8913,"mutex_wait_us":504,"spinlock_wait_cycles":42240,"thread_start_us":1943,"threads_started":1}
I20260806 19:42:50.052102 29583 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:50.053495 29581 heartbeater.cc:347] Connected to a master server at 127.21.202.124:46083
I20260806 19:42:50.053519 29583 heartbeater.cc:511] Master 127.21.202.125:44803 requested a full tablet report, sending...
I20260806 19:42:50.053655 29581 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:50.054148 29581 heartbeater.cc:511] Master 127.21.202.124:46083 requested a full tablet report, sending...
I20260806 19:42:50.052408 29587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.016269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.65:53191)
0806 19:42:50.025276 (+ 9007us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:50.025302 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:50.025460 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:50.027123 (+ 1663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:50.027127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:50.027785 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:50.028077 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:50.028088 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.030351 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.030356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:50.031963 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:50.031980 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.033471 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.033477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:50.033666 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.035055 (+ 1389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:50.035085 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:50.039418 (+ 4333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:50.044624 (+ 5206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:50.044629 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:50.044632 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:50.045175 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:50.045836 (+ 661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:50.045840 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:50.045843 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:50.046025 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:50.047042 (+ 1017us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:50.047047 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:50.047192 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 19:42:50.047331 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 19:42:50.047962 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8832,"mutex_wait_us":355,"thread_start_us":7069,"threads_started":1}
I20260806 19:42:50.055521 29582 heartbeater.cc:347] Connected to a master server at 127.21.202.126:45677
I20260806 19:42:50.055656 29582 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:50.056118 29582 heartbeater.cc:511] Master 127.21.202.126:45677 requested a full tablet report, sending...
I20260806 19:42:50.056109 29207 ts_manager.cc:194] Registered new tserver with Master: d64104c8ac3b4b4fb0f11c7d035d575c (127.21.202.65:36007)
I20260806 19:42:50.056237 29400 ts_manager.cc:194] Registered new tserver with Master: d64104c8ac3b4b4fb0f11c7d035d575c (127.21.202.65:36007)
I20260806 19:42:50.057354 28593 ts_manager.cc:194] Re-registered known tserver with Master: d64104c8ac3b4b4fb0f11c7d035d575c (127.21.202.65:36007)
I20260806 19:42:50.058552 28593 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:53191
I20260806 19:42:50.061764 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.013528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42860 (local address 127.21.202.66:33619)
0806 19:42:50.026489 (+ 12961us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:50.026497 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:50.027308 (+ 811us) server_negotiation.cc:407] Connection header received
0806 19:42:50.027394 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:50.027405 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:50.027657 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:50.027822 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:50.035202 (+ 7380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.036849 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.038759 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.044909 (+ 6150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.045054 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.053012 (+ 7958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:50.053031 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:50.053047 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:50.053102 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:50.058223 (+ 5121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.059364 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.059374 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.059377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.059555 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.060111 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.060119 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.060125 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.060554 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:50.060934 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:50.061257 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 19:42:50.061474 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12764,"thread_start_us":12591,"threads_started":1}
I20260806 19:42:50.063046 28916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:50.152254 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 28792
I20260806 19:42:50.170847 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:33619
--local_ip_for_outbound_sockets=127.21.202.66
--tserver_master_addrs=127.21.202.126:45677
--webserver_port=34485
--webserver_interface=127.21.202.66
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab
--principal=oryx/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:35153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.202.126:45677,127.21.202.125:44803
--tserver_master_addrs=127.21.202.126:45677,127.21.202.125:44803,127.21.202.124:46083 with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:50.525743 29593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:50.526077 29593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:50.526249 29593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:50.533746 29593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:50.533954 29593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:50.534133 29593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:42:50.545627 29593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:35153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab
--principal=oryx/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:33619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=34485
--tserver_master_addrs=127.21.202.126:45677,127.21.202.125:44803,127.21.202.124:46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:50.547142 29593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:50.548610 29593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:50.565732 29598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:50.565968 29599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:50.566131 29601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:50.568264 29593 server_base.cc:1034] running on GCE node
I20260806 19:42:50.569047 29593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:50.570066 29593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:50.571336 29593 hybrid_clock.cc:648] HybridClock initialized: now 1786045370571264 us; error 62 us; skew 500 ppm
I20260806 19:42:50.571776 29593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:50 dist-test-slave-77v9 krb5kdc[28549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045370, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:50.575736 29593 init.cc:377] Logged in from keytab as oryx/127.21.202.66@KRBTEST.COM (short username oryx)
I20260806 19:42:50.577622 29593 webserver.cc:460] Webserver started at http://127.21.202.66:34485/ using document root <none> and password file <none>
I20260806 19:42:50.578536 29593 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:50.578804 29593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:50.585325 29593 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:42:50.588641 29608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:50.590248 29593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:50.590440 29593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "d3bffa83f0eb452ab48524820f9e1aea"
format_stamp: "Formatted at 2026-08-06 19:42:47 on dist-test-slave-77v9"
server_key: "a9e88475725d964a2fa081072fc0bf48"
server_key_iv: "049998c93c76b38b5fd0a4c45f6bfcf6"
server_key_version: "encryptionkey@0"
I20260806 19:42:50.591568 29593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:50.615792 29593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:50.621224 29593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:50.621567 29593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:50.622762 29593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:50.624362 29593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:50.624464 29593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:50.624616 29593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:50.624730 29593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:50.663041 29593 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:33619
I20260806 19:42:50.663142 29721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:33619 every 8 connection(s)
I20260806 19:42:50.664750 29593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:42:50.668598 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 29593
I20260806 19:42:50.668995 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 28927
Aug 06 19:42:50 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045370, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.125@KRBTEST.COM
Aug 06 19:42:50 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045370, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
Aug 06 19:42:50 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.202.67@KRBTEST.COM
Aug 06 19:42:50 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045370, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.124@KRBTEST.COM
I20260806 19:42:50.710791 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.669847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:48255 (local address 127.21.202.125:44803)
0806 19:42:50.683563 (+ 13716us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:50.683570 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:50.683590 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:42:50.684517 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:50.684523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:50.684635 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:50.684750 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:50.687154 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.687960 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.690628 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.691428 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.691552 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.699806 (+ 8254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:50.699813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:50.699819 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:50.699869 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:50.705275 (+ 5406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.706813 (+ 1538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.706820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.706825 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.707011 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.707699 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.707707 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.707714 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.708064 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:50.708288 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:50.709986 (+ 1698us) server_negotiation.cc:299] Negotiation successful
0806 19:42:50.710294 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13571,"thread_start_us":13380,"threads_started":1}
I20260806 19:42:50.711503 29726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.669840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:44803 (local address 127.21.202.66:48255)
0806 19:42:50.682863 (+ 13023us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:50.682893 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:50.683973 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:50.685145 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:50.685155 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:50.685916 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:50.686891 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:50.686915 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.688128 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.688135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:50.690409 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:50.690427 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.691987 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.691997 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:50.692401 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.695434 (+ 3033us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:50.695468 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:50.699491 (+ 4023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:50.705475 (+ 5984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:50.705488 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:50.705514 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:50.706617 (+ 1103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:50.707175 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:50.707181 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:50.707186 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:50.707508 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:50.708237 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:50.708247 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:50.708719 (+ 472us) client_negotiation.cc:769] Sending connection context
0806 19:42:50.709178 (+ 459us) client_negotiation.cc:240] Negotiation successful
0806 19:42:50.709933 (+ 755us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12807,"thread_start_us":24326,"threads_started":1}
I20260806 19:42:50.713236 29724 heartbeater.cc:347] Connected to a master server at 127.21.202.125:44803
I20260806 19:42:50.713641 29724 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:50.714694 29724 heartbeater.cc:511] Master 127.21.202.125:44803 requested a full tablet report, sending...
I20260806 19:42:50.717895 29734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.672393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55906 (local address 127.21.202.67:42981)
0806 19:42:50.677719 (+ 5326us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:50.677735 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:50.689563 (+ 11828us) server_negotiation.cc:407] Connection header received
0806 19:42:50.689904 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:50.689914 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:50.690203 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:50.690379 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:50.693157 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.695058 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.699394 (+ 4336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.700460 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.700627 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.707781 (+ 7154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:50.707801 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:50.707826 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:50.707874 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:50.713185 (+ 5311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.714169 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.714178 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.714182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.714367 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.715088 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.715097 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.715104 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.715637 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:50.716044 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:50.717073 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0806 19:42:50.717399 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5043,"thread_start_us":4749,"threads_started":1}
I20260806 19:42:50.718070 29207 ts_manager.cc:194] Registered new tserver with Master: d3bffa83f0eb452ab48524820f9e1aea (127.21.202.66:33619)
I20260806 19:42:50.719208 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.669728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:55225 (local address 127.21.202.126:45677)
0806 19:42:50.693530 (+ 23802us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:50.693537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:50.693561 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:42:50.693625 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:50.693631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:50.693762 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:50.693893 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:50.695246 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.696478 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.699934 (+ 3456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.700409 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.700578 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.705148 (+ 4570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:50.705154 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:50.705158 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:50.705198 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:50.709065 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.714699 (+ 5634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.714719 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.714723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.715055 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.716771 (+ 1716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.716778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.716781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.717387 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:50.717570 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:50.718320 (+ 750us) server_negotiation.cc:299] Negotiation successful
0806 19:42:50.718594 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23652,"thread_start_us":23659,"threads_started":1}
I20260806 19:42:50.719391 29732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.669071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.66:55225)
0806 19:42:50.678585 (+ 9514us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:50.678675 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:50.688128 (+ 9453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:50.693991 (+ 5863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:50.694001 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:50.694646 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:50.695067 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:50.695082 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.696651 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.696656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:50.698638 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:50.698654 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.700630 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.700636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:50.700895 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.701571 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:50.701596 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:50.704891 (+ 3295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:50.713408 (+ 8517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:50.713414 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:50.713418 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:50.714034 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:50.715419 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:50.715425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:50.715430 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:50.715828 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:50.717522 (+ 1694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:50.717530 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:50.717700 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 19:42:50.717888 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 19:42:50.718948 (+ 1060us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9038,"spinlock_wait_cycles":49664,"thread_start_us":13245,"threads_started":1}
I20260806 19:42:50.720363 29723 heartbeater.cc:347] Connected to a master server at 127.21.202.126:45677
I20260806 19:42:50.720463 29052 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:50.720523 29723 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:50.720707 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.670056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:52877 (local address 127.21.202.124:46083)
0806 19:42:50.685697 (+ 15641us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:50.685707 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:50.694788 (+ 9081us) server_negotiation.cc:407] Connection header received
0806 19:42:50.694941 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:50.694950 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:50.695091 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:50.695215 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:50.698326 (+ 3111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.699760 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.702987 (+ 3227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.703504 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.703588 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.710256 (+ 6668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:50.710267 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:50.710271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:50.710317 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:50.713768 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.714694 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.714701 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.714704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.715036 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:50.715812 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:50.715818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:50.715822 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:50.716252 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:50.716539 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:50.719890 (+ 3351us) server_negotiation.cc:299] Negotiation successful
0806 19:42:50.720223 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15479,"thread_start_us":15420,"threads_started":1}
I20260806 19:42:50.721200 29723 heartbeater.cc:511] Master 127.21.202.126:45677 requested a full tablet report, sending...
I20260806 19:42:50.721966 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:50.669071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:46083 (local address 127.21.202.66:52877)
0806 19:42:50.694635 (+ 25564us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:50.694657 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:50.694804 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:50.696691 (+ 1887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:50.696696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:50.697628 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:50.698058 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:50.698105 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.700872 (+ 2767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.700878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:50.702748 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:50.702769 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:50.703947 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:50.703957 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:50.704340 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:50.705951 (+ 1611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:50.705977 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:50.709984 (+ 4007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:50.713944 (+ 3960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:50.713949 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:50.713952 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:50.714510 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:50.715354 (+ 844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:50.715359 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:50.715363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:50.715658 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:50.717669 (+ 2011us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:50.717674 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:50.717854 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 19:42:50.721023 (+ 3169us) client_negotiation.cc:240] Negotiation successful
0806 19:42:50.721393 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25383,"mutex_wait_us":994,"thread_start_us":7859,"threads_started":1}
I20260806 19:42:50.722555 28593 ts_manager.cc:194] Re-registered known tserver with Master: d3bffa83f0eb452ab48524820f9e1aea (127.21.202.66:33619)
I20260806 19:42:50.722848 29722 heartbeater.cc:347] Connected to a master server at 127.21.202.124:46083
I20260806 19:42:50.722978 29722 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:50.723371 29722 heartbeater.cc:511] Master 127.21.202.124:46083 requested a full tablet report, sending...
I20260806 19:42:50.723809 28593 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:55225
I20260806 19:42:50.724565 29400 ts_manager.cc:194] Registered new tserver with Master: d3bffa83f0eb452ab48524820f9e1aea (127.21.202.66:33619)
I20260806 19:42:50.823053 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 28927
I20260806 19:42:50.842971 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:42981
--local_ip_for_outbound_sockets=127.21.202.67
--tserver_master_addrs=127.21.202.126:45677
--webserver_port=43925
--webserver_interface=127.21.202.67
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab
--principal=oryx/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:35153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.202.126:45677,127.21.202.125:44803
--tserver_master_addrs=127.21.202.126:45677,127.21.202.125:44803,127.21.202.124:46083 with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:42:51.061427 29582 heartbeater.cc:503] Master 127.21.202.126:45677 was elected leader, sending a full tablet report...
W20260806 19:42:51.199257 29736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:51.199611 29736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:51.199796 29736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:51.207585 29736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:51.207753 29736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:51.207935 29736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:42:51.218220 29736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:35153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab
--principal=oryx/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:42981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=43925
--tserver_master_addrs=127.21.202.126:45677,127.21.202.125:44803,127.21.202.124:46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:51.219637 29736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:51.221547 29736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:51.236836 29744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:51.236923 29741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:51.236954 29742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:51.237948 29736 server_base.cc:1034] running on GCE node
I20260806 19:42:51.238718 29736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:51.241111 29736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:51.242372 29736 hybrid_clock.cc:648] HybridClock initialized: now 1786045371242329 us; error 32 us; skew 500 ppm
I20260806 19:42:51.242786 29736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:51 dist-test-slave-77v9 krb5kdc[28549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045371, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:51.246446 29736 init.cc:377] Logged in from keytab as oryx/127.21.202.67@KRBTEST.COM (short username oryx)
I20260806 19:42:51.248003 29736 webserver.cc:460] Webserver started at http://127.21.202.67:43925/ using document root <none> and password file <none>
I20260806 19:42:51.248713 29736 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:51.248925 29736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:51.254565 29736 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 19:42:51.257483 29751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:51.258940 29736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:42:51.259130 29736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "0a5049e41f514390804e74e692bc90bf"
format_stamp: "Formatted at 2026-08-06 19:42:47 on dist-test-slave-77v9"
server_key: "2a926aa3967a6b505a188ab057f022e4"
server_key_iv: "a0362585e8b450738c2c7ee80cdf2b7f"
server_key_version: "encryptionkey@0"
I20260806 19:42:51.260236 29736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:51.279487 29736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:51.284267 29736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:51.284600 29736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:51.285781 29736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:42:51.287344 29736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:51.287468 29736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:51.287604 29736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:51.287694 29736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:51.325786 29736 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:42981
I20260806 19:42:51.325874 29863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:42981 every 8 connection(s)
I20260806 19:42:51.327411 29736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:42:51.331805 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 29736
Aug 06 19:42:51 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045371, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
Aug 06 19:42:51 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045371, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.125@KRBTEST.COM
Aug 06 19:42:51 dist-test-slave-77v9 krb5kdc[28549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045371, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.124@KRBTEST.COM
I20260806 19:42:51.367795 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.333918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:39495 (local address 127.21.202.124:46083)
0806 19:42:51.343734 (+ 9816us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.343743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.343772 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:42:51.343851 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.343859 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.343963 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.345769 (+ 1806us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:51.345958 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.347304 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.350544 (+ 3240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.351518 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.352567 (+ 1049us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.359510 (+ 6943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:51.359518 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:51.359523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:51.359573 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:51.364050 (+ 4477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.365149 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.365154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.365159 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.365362 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.365869 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.365874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.365877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.366231 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:51.366388 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.367138 (+ 750us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.367386 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9640,"thread_start_us":9532,"threads_started":1}
I20260806 19:42:51.367995 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.331105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:45047 (local address 127.21.202.126:45677)
0806 19:42:51.341209 (+ 10104us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.341214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.343434 (+ 2220us) server_negotiation.cc:407] Connection header received
0806 19:42:51.343506 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.343510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.343592 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.343694 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:51.345327 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.346444 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.348935 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.349652 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.349759 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.355681 (+ 5922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:51.355692 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:51.355698 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:51.355764 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:51.361082 (+ 5318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.363163 (+ 2081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.363172 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.363178 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.363396 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.365096 (+ 1700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.365105 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.365111 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.365666 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:51.365858 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.366968 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.367374 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9990,"thread_start_us":9898,"threads_started":1}
I20260806 19:42:51.369037 29871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.331924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.67:45047)
0806 19:42:51.343131 (+ 11207us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:51.343155 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:51.343270 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:51.343870 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:51.343880 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:51.344701 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:51.345152 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.345165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.346675 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.346680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.348707 (+ 2027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.348725 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.350441 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.350450 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.350669 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.351505 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:51.351553 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:51.355358 (+ 3805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:51.362123 (+ 6765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:51.362136 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:51.362159 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:51.362985 (+ 826us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:51.364744 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:51.364750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:51.364753 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:51.364962 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:51.365834 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:51.365846 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:51.366374 (+ 528us) client_negotiation.cc:769] Sending connection context
0806 19:42:51.366792 (+ 418us) client_negotiation.cc:240] Negotiation successful
0806 19:42:51.367420 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11091,"mutex_wait_us":40,"thread_start_us":10183,"threads_started":1}
I20260806 19:42:51.369663 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.331239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:46083 (local address 127.21.202.67:39495)
0806 19:42:51.342415 (+ 11176us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:51.342453 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:51.342580 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:51.344216 (+ 1636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:51.344224 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:51.345199 (+ 975us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:51.345600 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.345615 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.347544 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.347557 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.350172 (+ 2615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.350188 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.351844 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.351854 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.352108 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.352887 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:51.352917 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:51.358425 (+ 5508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:51.364282 (+ 5857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:51.364288 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:51.364292 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:51.365000 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:51.365516 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:51.365521 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:51.365527 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:51.365707 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:51.366489 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:51.366495 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:51.366653 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 19:42:51.366792 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 19:42:51.368834 (+ 2042us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10997,"mutex_wait_us":1665,"spinlock_wait_cycles":56960,"thread_start_us":5495,"threads_started":1}
I20260806 19:42:51.370860 29866 heartbeater.cc:347] Connected to a master server at 127.21.202.126:45677
I20260806 19:42:51.371220 29866 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:51.371601 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.330656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:40187 (local address 127.21.202.125:44803)
0806 19:42:51.341520 (+ 10864us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.341524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.341541 (+ 17us) server_negotiation.cc:407] Connection header received
0806 19:42:51.341596 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.341600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.341671 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.341803 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:51.344458 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.345721 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.348777 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.349542 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.349663 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.357026 (+ 7363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:51.357032 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:51.357036 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:51.357085 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:51.361627 (+ 4542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.363909 (+ 2282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.363915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.363918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.364096 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.365138 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.365144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.365147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.365602 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:51.365740 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.370913 (+ 5173us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.371191 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10764,"thread_start_us":10674,"threads_started":1}
I20260806 19:42:51.372197 29866 heartbeater.cc:511] Master 127.21.202.126:45677 requested a full tablet report, sending...
I20260806 19:42:51.374063 29872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.331239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:44803 (local address 127.21.202.67:40187)
0806 19:42:51.337369 (+ 6130us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:51.337417 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:51.339288 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:51.342142 (+ 2854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:51.342157 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:51.343214 (+ 1057us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:51.344184 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.344216 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.345941 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.345946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.348535 (+ 2589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.348552 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.350106 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.350115 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.350496 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.351590 (+ 1094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:51.351612 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:51.356154 (+ 4542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:51.363181 (+ 7027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:51.363188 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:51.363192 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:51.363734 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:51.364282 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:51.364288 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:51.364292 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:51.364600 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:51.369797 (+ 5197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:51.369805 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:51.369976 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 19:42:51.371802 (+ 1826us) client_negotiation.cc:240] Negotiation successful
0806 19:42:51.372158 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5809,"mutex_wait_us":1,"thread_start_us":11440,"threads_started":1}
I20260806 19:42:51.374194 28593 ts_manager.cc:194] Re-registered known tserver with Master: 0a5049e41f514390804e74e692bc90bf (127.21.202.67:42981)
I20260806 19:42:51.375768 28593 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:45047
I20260806 19:42:51.376983 29864 heartbeater.cc:347] Connected to a master server at 127.21.202.124:46083
I20260806 19:42:51.377193 29864 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:51.377866 29864 heartbeater.cc:511] Master 127.21.202.124:46083 requested a full tablet report, sending...
I20260806 19:42:51.379738 29867 heartbeater.cc:347] Connected to a master server at 127.21.202.125:44803
I20260806 19:42:51.379832 29400 ts_manager.cc:194] Registered new tserver with Master: 0a5049e41f514390804e74e692bc90bf (127.21.202.67:42981)
I20260806 19:42:51.379902 29867 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:51.380352 29867 heartbeater.cc:511] Master 127.21.202.125:44803 requested a full tablet report, sending...
I20260806 19:42:51.381851 29207 ts_manager.cc:194] Registered new tserver with Master: 0a5049e41f514390804e74e692bc90bf (127.21.202.67:42981)
I20260806 19:42:51.393276 29880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.358892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.21.202.124:46083)
0806 19:42:51.364081 (+ 5189us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.364095 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.364432 (+ 337us) server_negotiation.cc:407] Connection header received
0806 19:42:51.364500 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.364509 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.364706 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.364853 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:51.371012 (+ 6159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.372520 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.375368 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.380426 (+ 5058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.384087 (+ 3661us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.384373 (+ 286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:51.384378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:51.384382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:51.384408 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:51.388640 (+ 4232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.389604 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.389610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.389616 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.389803 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.390385 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.390394 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.390399 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.390951 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:51.391250 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.392521 (+ 1271us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.392766 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4999,"thread_start_us":4843,"threads_started":1}
I20260806 19:42:51.399545 29882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.359266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41412 (local address 127.21.202.125:44803)
0806 19:42:51.361793 (+ 2527us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.361802 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.379202 (+ 17400us) server_negotiation.cc:407] Connection header received
0806 19:42:51.379371 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.379376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.379466 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.379575 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:51.383814 (+ 4239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.385352 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.387415 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.387917 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.388051 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.392345 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:51.392354 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:51.392362 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:51.392415 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:51.396364 (+ 3949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.397172 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.397178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.397182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.397360 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.397777 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.397782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.397786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.398114 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:51.398433 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.398786 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.399040 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2321,"thread_start_us":2136,"threads_started":1}
I20260806 19:42:51.400353 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.362232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45370 (local address 127.21.202.126:45677)
0806 19:42:51.368295 (+ 6063us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.368303 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.381164 (+ 12861us) server_negotiation.cc:407] Connection header received
0806 19:42:51.381910 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.381920 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.382059 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.382185 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:51.386014 (+ 3829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.387291 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.389675 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.390192 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.390554 (+ 362us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.394425 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:51.394431 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:51.394434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:51.394474 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:51.397553 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.398251 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.398264 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.398266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.398399 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.398793 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.398797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.398799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.399113 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:51.399352 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.399801 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.400018 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5925,"thread_start_us":20029,"threads_started":1}
I20260806 19:42:51.405121 28593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:42:51.412482 28593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:42:51.457404 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.436264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55918 (local address 127.21.202.67:42981)
0806 19:42:51.449011 (+ 12747us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.449025 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.449069 (+ 44us) server_negotiation.cc:407] Connection header received
0806 19:42:51.449195 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.449206 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.449537 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.449788 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:51.451234 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.452617 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.454608 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.456202 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.456334 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.456517 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.456916 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.457073 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12339,"thread_start_us":12132,"threads_started":1}
I20260806 19:42:51.457520 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.434452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:42981 (local address 127.0.0.1:55918)
0806 19:42:51.435255 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:51.435276 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:51.435466 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:51.449784 (+ 14318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:51.449794 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:51.449834 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:51.450265 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.450279 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.452739 (+ 2460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.452745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.454441 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.454454 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.456420 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.456426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.456670 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.456709 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 19:42:51.456855 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 19:42:51.457000 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":31,"thread_start_us":416,"threads_started":1}
I20260806 19:42:51.458148 29898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.435228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:33619 (local address 127.0.0.1:42864)
0806 19:42:51.449803 (+ 14575us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:51.449821 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:51.449957 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:51.450764 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:51.450769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:51.450791 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:51.451272 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.451293 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.453201 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.453205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.454755 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.454771 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.457404 (+ 2633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.457412 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.457618 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.457665 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 19:42:51.457752 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 19:42:51.457885 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14419,"thread_start_us":14246,"threads_started":1}
I20260806 19:42:51.458304 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.435765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42864 (local address 127.21.202.66:33619)
0806 19:42:51.437113 (+ 1348us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.437123 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.449939 (+ 12816us) server_negotiation.cc:407] Connection header received
0806 19:42:51.450307 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.450316 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.450535 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.450742 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:51.451475 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.453062 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.454957 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.455924 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.456025 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.456277 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.457877 (+ 1600us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.458059 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1089,"thread_start_us":1078,"threads_started":1}
I20260806 19:42:51.461212 29900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.435915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43278 (local address 127.21.202.65:36007)
0806 19:42:51.444263 (+ 8348us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.444272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.450192 (+ 5920us) server_negotiation.cc:407] Connection header received
0806 19:42:51.450310 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.450317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.450586 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.450755 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:51.451728 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.454941 (+ 3213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.458363 (+ 3422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.460036 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.460181 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.460381 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.460730 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.460939 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8169,"thread_start_us":8057,"threads_started":1}
I20260806 19:42:51.461237 29897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.435356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:36007 (local address 127.0.0.1:43278)
0806 19:42:51.449876 (+ 14520us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:51.449900 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:51.450048 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:51.450992 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:51.450996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:51.451026 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:51.451550 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.451562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.455078 (+ 3516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.455083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.457144 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.457159 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.460346 (+ 3187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.460354 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.460531 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.460566 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 19:42:51.460671 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 19:42:51.460889 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14377,"mutex_wait_us":3,"thread_start_us":13733,"threads_started":1}
I20260806 19:42:51.461864 29799 tablet_service.cc:1467] Processing CreateTablet for tablet 36a4a129822245fbb6d308f87c50c3c8 (DEFAULT_TABLE table=test-table [id=2da0ad7614a7432a8abb0eb925c778a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:51.461894 29657 tablet_service.cc:1467] Processing CreateTablet for tablet 36a4a129822245fbb6d308f87c50c3c8 (DEFAULT_TABLE table=test-table [id=2da0ad7614a7432a8abb0eb925c778a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:51.463191 29657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36a4a129822245fbb6d308f87c50c3c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:51.463261 29799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36a4a129822245fbb6d308f87c50c3c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:51.465363 29514 tablet_service.cc:1467] Processing CreateTablet for tablet 36a4a129822245fbb6d308f87c50c3c8 (DEFAULT_TABLE table=test-table [id=2da0ad7614a7432a8abb0eb925c778a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:51.466984 29514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36a4a129822245fbb6d308f87c50c3c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:51.478771 29902 tablet_bootstrap.cc:492] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea: Bootstrap starting.
I20260806 19:42:51.482194 29903 tablet_bootstrap.cc:492] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c: Bootstrap starting.
I20260806 19:42:51.483024 29902 tablet_bootstrap.cc:654] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:51.483501 29904 tablet_bootstrap.cc:492] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf: Bootstrap starting.
I20260806 19:42:51.485373 29902 log.cc:824] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:51.486387 29903 tablet_bootstrap.cc:654] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:51.486929 29904 tablet_bootstrap.cc:654] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:51.487792 29903 log.cc:824] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:51.487870 29904 log.cc:824] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:51.489102 29902 tablet_bootstrap.cc:492] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea: No bootstrap required, opened a new log
I20260806 19:42:51.489446 29902 ts_tablet_manager.cc:1397] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260806 19:42:51.491382 29904 tablet_bootstrap.cc:492] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf: No bootstrap required, opened a new log
I20260806 19:42:51.491472 29903 tablet_bootstrap.cc:492] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c: No bootstrap required, opened a new log
I20260806 19:42:51.491667 29904 ts_tablet_manager.cc:1397] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 19:42:51.491747 29903 ts_tablet_manager.cc:1397] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 19:42:51.499528 29904 raft_consensus.cc:348] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } } peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } }
I20260806 19:42:51.499964 29904 raft_consensus.cc:374] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:51.500069 29904 raft_consensus.cc:729] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a5049e41f514390804e74e692bc90bf, State: Initialized, Role: FOLLOWER
I20260806 19:42:51.500723 29904 consensus_queue.cc:260] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } } peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } }
I20260806 19:42:51.501631 29902 raft_consensus.cc:348] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } } peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } }
I20260806 19:42:51.502118 29902 raft_consensus.cc:374] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:51.502214 29902 raft_consensus.cc:729] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bffa83f0eb452ab48524820f9e1aea, State: Initialized, Role: FOLLOWER
I20260806 19:42:51.502691 29904 ts_tablet_manager.cc:1428] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260806 19:42:51.503006 29866 heartbeater.cc:503] Master 127.21.202.126:45677 was elected leader, sending a full tablet report...
I20260806 19:42:51.502840 29902 consensus_queue.cc:260] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } } peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } }
I20260806 19:42:51.504087 29903 raft_consensus.cc:348] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } } peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } }
I20260806 19:42:51.504621 29903 raft_consensus.cc:374] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:51.504726 29903 raft_consensus.cc:729] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64104c8ac3b4b4fb0f11c7d035d575c, State: Initialized, Role: FOLLOWER
I20260806 19:42:51.505439 29903 consensus_queue.cc:260] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } } peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } }
I20260806 19:42:51.505831 29902 ts_tablet_manager.cc:1428] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260806 19:42:51.506168 29723 heartbeater.cc:503] Master 127.21.202.126:45677 was elected leader, sending a full tablet report...
I20260806 19:42:51.513397 29903 ts_tablet_manager.cc:1428] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c: Time spent starting tablet: real 0.022s user 0.019s sys 0.000s
W20260806 19:42:51.525560 29585 tablet.cc:2369] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:42:51.551591 29908 raft_consensus.cc:482] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:51.551968 29908 raft_consensus.cc:504] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } } peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } }
I20260806 19:42:51.553936 29908 leader_election.cc:288] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3bffa83f0eb452ab48524820f9e1aea (127.21.202.66:33619), d64104c8ac3b4b4fb0f11c7d035d575c (127.21.202.65:36007)
I20260806 19:42:51.561836 29900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.554627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:51095 (local address 127.21.202.65:36007)
0806 19:42:51.554858 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.554867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.554897 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:42:51.555003 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.555009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.555106 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.555221 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:51.555667 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.556556 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.559317 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.560743 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.560967 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.561070 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.561384 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.561537 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 19:42:51.561852 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.554207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:36007 (local address 127.21.202.67:51095)
0806 19:42:51.554509 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:51.554527 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:51.554668 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:51.555252 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:51.555258 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:51.555283 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:51.555506 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.555516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.556716 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.556722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.559057 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.559075 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.560949 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.560955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.561195 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.561232 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:42:51.561330 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 19:42:51.561491 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":1,"spinlock_wait_cycles":17792}
I20260806 19:42:51.562434 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.554876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:53637 (local address 127.21.202.66:33619)
0806 19:42:51.555107 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.555114 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.555134 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:42:51.555213 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.555218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.555292 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.555385 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:51.555880 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.556986 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.559486 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.560963 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.561104 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.561186 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.561999 (+ 813us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.562142 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 19:42:51.562958 29872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.554118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:33619 (local address 127.21.202.67:53637)
0806 19:42:51.554503 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:51.554528 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:51.554669 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:51.555458 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:51.555463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:51.555493 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:51.555698 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.555713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.557140 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.557145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.559270 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:51.559283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.561234 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.561240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:51.561502 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.561538 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 19:42:51.561645 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 19:42:51.561780 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"spinlock_wait_cycles":5504}
I20260806 19:42:51.563678 29534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a4a129822245fbb6d308f87c50c3c8" candidate_uuid: "0a5049e41f514390804e74e692bc90bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" is_pre_election: true
I20260806 19:42:51.564062 29677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a4a129822245fbb6d308f87c50c3c8" candidate_uuid: "0a5049e41f514390804e74e692bc90bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bffa83f0eb452ab48524820f9e1aea" is_pre_election: true
I20260806 19:42:51.564247 29534 raft_consensus.cc:2454] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a5049e41f514390804e74e692bc90bf in term 0.
I20260806 19:42:51.564662 29677 raft_consensus.cc:2454] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a5049e41f514390804e74e692bc90bf in term 0.
I20260806 19:42:51.565114 29755 leader_election.cc:302] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a5049e41f514390804e74e692bc90bf, d64104c8ac3b4b4fb0f11c7d035d575c; no voters:
I20260806 19:42:51.565666 29908 raft_consensus.cc:2781] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:42:51.565856 29908 raft_consensus.cc:482] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:51.565991 29908 raft_consensus.cc:3037] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:51.568251 29908 raft_consensus.cc:504] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } } peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } }
I20260806 19:42:51.568948 29908 leader_election.cc:288] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [CANDIDATE]: Term 1 election: Requested vote from peers d3bffa83f0eb452ab48524820f9e1aea (127.21.202.66:33619), d64104c8ac3b4b4fb0f11c7d035d575c (127.21.202.65:36007)
I20260806 19:42:51.569486 29534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a4a129822245fbb6d308f87c50c3c8" candidate_uuid: "0a5049e41f514390804e74e692bc90bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c"
I20260806 19:42:51.569628 29677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a4a129822245fbb6d308f87c50c3c8" candidate_uuid: "0a5049e41f514390804e74e692bc90bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bffa83f0eb452ab48524820f9e1aea"
I20260806 19:42:51.569734 29534 raft_consensus.cc:3037] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:51.569836 29677 raft_consensus.cc:3037] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:51.573156 29534 raft_consensus.cc:2454] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a5049e41f514390804e74e692bc90bf in term 1.
I20260806 19:42:51.573156 29677 raft_consensus.cc:2454] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a5049e41f514390804e74e692bc90bf in term 1.
I20260806 19:42:51.573634 29754 leader_election.cc:302] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a5049e41f514390804e74e692bc90bf, d3bffa83f0eb452ab48524820f9e1aea; no voters:
I20260806 19:42:51.573966 29908 raft_consensus.cc:2781] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:51.574481 29908 raft_consensus.cc:686] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [term 1 LEADER]: Becoming Leader. State: Replica: 0a5049e41f514390804e74e692bc90bf, State: Running, Role: LEADER
I20260806 19:42:51.575387 29908 consensus_queue.cc:237] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } } peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } }
I20260806 19:42:51.581167 28594 catalog_manager.cc:5495] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a5049e41f514390804e74e692bc90bf (127.21.202.67). New cstate: current_term: 1 leader_uuid: "0a5049e41f514390804e74e692bc90bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } health_report { overall_health: UNKNOWN } } }
W20260806 19:42:51.586578 29868 tablet.cc:2369] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:42:51.601457 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.596029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55922 (local address 127.21.202.67:42981)
0806 19:42:51.596266 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.596272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.596293 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:42:51.596463 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.596467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.596554 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.596707 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:51.597082 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.597798 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.599225 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.599648 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.599750 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.600256 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:51.600881 (+ 625us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:51.601006 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.601166 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.601238 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 19:42:51.610005 29677 raft_consensus.cc:1261] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea [term 1 FOLLOWER]: Refusing update from remote peer 0a5049e41f514390804e74e692bc90bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:51.610005 29534 raft_consensus.cc:1261] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c [term 1 FOLLOWER]: Refusing update from remote peer 0a5049e41f514390804e74e692bc90bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:51.611058 29911 consensus_queue.cc:1035] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:51.611449 29908 consensus_queue.cc:1035] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:51.629664 29915 mvcc.cc:205] Tried to move back new op lower bound from 7315641842103562240 to 7315641841978109952. Current Snapshot: MvccSnapshot[applied={T|T < 7315641842103562240}]
I20260806 19:42:51.633239 29917 mvcc.cc:205] Tried to move back new op lower bound from 7315641842103562240 to 7315641841978109952. Current Snapshot: MvccSnapshot[applied={T|T < 7315641842103562240}]
I20260806 19:42:51.646741 28594 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:51.651654 28594 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:51.662261 29514 tablet_service.cc:1514] Processing DeleteTablet for tablet 36a4a129822245fbb6d308f87c50c3c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:51 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:43278
I20260806 19:42:51.662475 28594 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ec97e781650a49da8405e8eff3b5242e: Sending DeleteTablet for 3 replicas of tablet 36a4a129822245fbb6d308f87c50c3c8
I20260806 19:42:51.663669 29799 tablet_service.cc:1514] Processing DeleteTablet for tablet 36a4a129822245fbb6d308f87c50c3c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:51 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:55918
I20260806 19:42:51.667613 29513 tablet_service.cc:1514] Processing DeleteTablet for tablet 36a4a129822245fbb6d308f87c50c3c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:51 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:43278
I20260806 19:42:51.669723 29657 tablet_service.cc:1514] Processing DeleteTablet for tablet 36a4a129822245fbb6d308f87c50c3c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:51 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:42864
I20260806 19:42:51.674244 29924 tablet_replica.cc:330] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf: stopping tablet replica
I20260806 19:42:51.674791 29924 raft_consensus.cc:2229] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:42:51.675592 29924 raft_consensus.cc:2258] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:51.678114 29924 ts_tablet_manager.cc:1905] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:51.680032 29923 tablet_replica.cc:330] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c: stopping tablet replica
I20260806 19:42:51.680673 29923 raft_consensus.cc:2229] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:51.681306 29923 raft_consensus.cc:2258] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:51.685460 29923 ts_tablet_manager.cc:1905] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:51.688310 29924 ts_tablet_manager.cc:1918] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:51.688723 29924 log.cc:1196] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/36a4a129822245fbb6d308f87c50c3c8
I20260806 19:42:51.689615 29924 ts_tablet_manager.cc:1939] T 36a4a129822245fbb6d308f87c50c3c8 P 0a5049e41f514390804e74e692bc90bf: Deleting consensus metadata
I20260806 19:42:51.692189 28580 catalog_manager.cc:4841] TS 0a5049e41f514390804e74e692bc90bf (127.21.202.67:42981): tablet 36a4a129822245fbb6d308f87c50c3c8 (table test-table [id=2da0ad7614a7432a8abb0eb925c778a3]) successfully deleted
I20260806 19:42:51.693220 29923 ts_tablet_manager.cc:1918] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:51.693415 29923 log.cc:1196] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/36a4a129822245fbb6d308f87c50c3c8
I20260806 19:42:51.694092 29923 ts_tablet_manager.cc:1939] T 36a4a129822245fbb6d308f87c50c3c8 P d64104c8ac3b4b4fb0f11c7d035d575c: Deleting consensus metadata
W20260806 19:42:51.696954 28581 catalog_manager.cc:4808] TS d64104c8ac3b4b4fb0f11c7d035d575c (127.21.202.65:36007): delete failed for tablet 36a4a129822245fbb6d308f87c50c3c8 because the tablet was not found. No further retry: Not found: Tablet not found: 36a4a129822245fbb6d308f87c50c3c8
I20260806 19:42:51.701388 29926 tablet_replica.cc:330] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea: stopping tablet replica
I20260806 19:42:51.701885 29926 raft_consensus.cc:2229] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:51.703402 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 28562
I20260806 19:42:51.702816 29926 raft_consensus.cc:2258] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:51.704586 28624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:51.706288 29926 ts_tablet_manager.cc:1905] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:51.714298 29926 ts_tablet_manager.cc:1918] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:51.714514 29926 log.cc:1196] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/36a4a129822245fbb6d308f87c50c3c8
I20260806 19:42:51.715129 29926 ts_tablet_manager.cc:1939] T 36a4a129822245fbb6d308f87c50c3c8 P d3bffa83f0eb452ab48524820f9e1aea: Deleting consensus metadata
I20260806 19:42:51.717208 28580 catalog_manager.cc:4841] TS d3bffa83f0eb452ab48524820f9e1aea (127.21.202.66:33619): tablet 36a4a129822245fbb6d308f87c50c3c8 (table test-table [id=2da0ad7614a7432a8abb0eb925c778a3]) successfully deleted
I20260806 19:42:51.729310 29882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.669240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41416 (local address 127.21.202.125:44803)
0806 19:42:51.669469 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.669476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.702665 (+ 33189us) server_negotiation.cc:407] Connection header received
0806 19:42:51.703048 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.703056 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.703150 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.703248 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:51.708038 (+ 4790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.709259 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.715514 (+ 6255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.716267 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.716419 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.721631 (+ 5212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:51.721639 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:51.721643 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:51.721694 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:51.725712 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.726771 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.726777 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.726781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.726937 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.727412 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.727418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.727421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.727808 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:51.728052 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.728689 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.728970 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 19:42:51.748775 29880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.702664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55574 (local address 127.21.202.124:46083)
0806 19:42:51.702882 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.702891 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.730331 (+ 27440us) server_negotiation.cc:407] Connection header received
0806 19:42:51.730512 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.730517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.730648 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.730752 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:51.733677 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.734864 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.737038 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.737747 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.737904 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.741815 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:51.741824 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:51.741830 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:51.741883 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:51.745623 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.746542 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.746548 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.746552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.746694 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.747136 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.747141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.747145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.747531 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:51.747793 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.748187 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.748421 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 19:42:51.866148 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 28562
I20260806 19:42:51.915597 29882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.895711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.21.202.125:44803)
0806 19:42:51.895934 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.895940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.897345 (+ 1405us) server_negotiation.cc:407] Connection header received
0806 19:42:51.897702 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.897712 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.897846 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.897941 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:51.901061 (+ 3120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.902613 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.904542 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.905295 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.905416 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.909170 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:51.909175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:51.909179 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:51.909217 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:51.912271 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.913046 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.913053 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.913056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.913189 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.913623 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.913653 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.913659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.914067 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:51.914313 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.914723 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.915169 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 19:42:51.920177 29880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:51.895583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55586 (local address 127.21.202.124:46083)
0806 19:42:51.895815 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:51.895823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:51.903168 (+ 7345us) server_negotiation.cc:407] Connection header received
0806 19:42:51.904154 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:51.904160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:51.904260 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:51.904394 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:51.906409 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.907546 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.909605 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:51.910256 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:51.910415 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:51.913788 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:51.913795 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:51.913800 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:51.913842 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:51.917420 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.918064 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.918070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.918074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.918217 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:51.918625 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:51.918631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:51.918634 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:51.919034 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:51.919273 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:51.919579 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 19:42:51.919828 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 19:42:52.639672 29935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:52.638529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.67:56983)
0806 19:42:52.639057 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:52.639246 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":251,"threads_started":1}
W20260806 19:42:52.640252 29866 heartbeater.cc:644] Failed to heartbeat to 127.21.202.126:45677 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
I20260806 19:42:52.652915 29936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:52.652010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.66:33483)
0806 19:42:52.652482 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:52.652665 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":229,"threads_started":1}
W20260806 19:42:52.653354 29723 heartbeater.cc:644] Failed to heartbeat to 127.21.202.126:45677 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
I20260806 19:42:52.663930 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:52.663033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.65:34347)
0806 19:42:52.663559 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:52.663750 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":288,"threads_started":1}
W20260806 19:42:52.664495 29582 heartbeater.cc:644] Failed to heartbeat to 127.21.202.126:45677 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
I20260806 19:42:53.161091 29938 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec97e781650a49da8405e8eff3b5242e)
I20260806 19:42:53.161384 29938 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } }
I20260806 19:42:53.162271 29939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec97e781650a49da8405e8eff3b5242e)
I20260806 19:42:53.162554 29939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } }
I20260806 19:42:53.163045 29938 leader_election.cc:288] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec97e781650a49da8405e8eff3b5242e (127.21.202.126:45677), 8ad649fbf617413fa2b842dc829219f9 (127.21.202.125:44803)
I20260806 19:42:53.164407 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.163153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:45814)
0806 19:42:53.163850 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:53.164040 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":410,"threads_started":1}
W20260806 19:42:53.164875 29386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
I20260806 19:42:53.166532 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.166066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:45826)
0806 19:42:53.166273 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:53.166354 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260806 19:42:53.166851 29386 leader_election.cc:334] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ec97e781650a49da8405e8eff3b5242e (127.21.202.126:45677): Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
I20260806 19:42:53.167376 29939 leader_election.cc:288] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec97e781650a49da8405e8eff3b5242e (127.21.202.126:45677), cfedfa797c6b45538571329bd5c9b53c (127.21.202.124:46083)
I20260806 19:42:53.168725 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.164108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:45820)
0806 19:42:53.168039 (+ 3931us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:53.168379 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3774,"thread_start_us":3635,"threads_started":1}
W20260806 19:42:53.169202 29194 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
I20260806 19:42:53.171496 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.171105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:45828)
0806 19:42:53.171266 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:53.171356 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 19:42:53.171684 29194 leader_election.cc:334] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ec97e781650a49da8405e8eff3b5242e (127.21.202.126:45677): Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
I20260806 19:42:53.177204 29942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.163843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41432 (local address 127.21.202.125:44803)
0806 19:42:53.170035 (+ 6192us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:53.170042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:53.170066 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:42:53.170133 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:53.170139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:53.170208 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:53.170335 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:53.171099 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:53.172368 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:53.174748 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:53.175984 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:53.176250 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:53.176325 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:53.176619 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 19:42:53.176760 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":5996,"thread_start_us":5788,"threads_started":1}
I20260806 19:42:53.177218 29941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.163397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:44803 (local address 127.0.0.1:41432)
0806 19:42:53.164784 (+ 1387us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:53.164803 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:53.165030 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:53.170390 (+ 5360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:53.170398 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:53.170434 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:53.170873 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:53.170893 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:53.172510 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:53.172516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:53.174599 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:53.174611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:53.176235 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:53.176242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:53.176439 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:53.176483 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 19:42:53.176588 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 19:42:53.176738 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1259,"thread_start_us":1129,"threads_started":1}
I20260806 19:42:53.177248 29946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.166502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55594 (local address 127.21.202.124:46083)
0806 19:42:53.169529 (+ 3027us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:53.169538 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:53.169568 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:42:53.169645 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:53.169652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:53.169747 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:53.170427 (+ 680us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:53.170751 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:53.172090 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:53.174579 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:53.175813 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:53.176021 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:53.176117 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:53.176619 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 19:42:53.176761 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":2860,"thread_start_us":2724,"threads_started":1}
I20260806 19:42:53.177361 29945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.166077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:46083 (local address 127.0.0.1:55594)
0806 19:42:53.168550 (+ 2473us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:53.168575 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:53.168754 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:53.169918 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:53.169936 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:53.169997 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:53.170357 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:53.170370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:53.172229 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:53.172233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:53.174403 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:53.174419 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:53.176057 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:53.176096 (+ 39us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:53.176379 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:53.176423 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 19:42:53.176545 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 19:42:53.176870 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2326,"mutex_wait_us":39,"thread_start_us":2035,"threads_started":1}
I20260806 19:42:53.178198 29217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfedfa797c6b45538571329bd5c9b53c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8ad649fbf617413fa2b842dc829219f9" is_pre_election: true
I20260806 19:42:53.178301 29410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8ad649fbf617413fa2b842dc829219f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfedfa797c6b45538571329bd5c9b53c" is_pre_election: true
I20260806 19:42:53.178614 29217 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfedfa797c6b45538571329bd5c9b53c in term 1.
I20260806 19:42:53.178654 29410 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ad649fbf617413fa2b842dc829219f9 in term 1.
I20260806 19:42:53.179188 29385 leader_election.cc:302] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ad649fbf617413fa2b842dc829219f9, cfedfa797c6b45538571329bd5c9b53c; no voters: ec97e781650a49da8405e8eff3b5242e
I20260806 19:42:53.179589 29938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 19:42:53.179761 29938 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec97e781650a49da8405e8eff3b5242e)
I20260806 19:42:53.179865 29938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 1 FOLLOWER]: Advancing to term 2
I20260806 19:42:53.179997 29191 leader_election.cc:302] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ad649fbf617413fa2b842dc829219f9, cfedfa797c6b45538571329bd5c9b53c; no voters: ec97e781650a49da8405e8eff3b5242e
I20260806 19:42:53.180408 29939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 19:42:53.180541 29939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec97e781650a49da8405e8eff3b5242e)
I20260806 19:42:53.180644 29939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 1 FOLLOWER]: Advancing to term 2
I20260806 19:42:53.183203 29938 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } }
I20260806 19:42:53.183993 29938 leader_election.cc:288] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [CANDIDATE]: Term 2 election: Requested vote from peers ec97e781650a49da8405e8eff3b5242e (127.21.202.126:45677), 8ad649fbf617413fa2b842dc829219f9 (127.21.202.125:44803)
I20260806 19:42:53.183859 29939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } }
I20260806 19:42:53.184536 29939 leader_election.cc:288] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [CANDIDATE]: Term 2 election: Requested vote from peers ec97e781650a49da8405e8eff3b5242e (127.21.202.126:45677), cfedfa797c6b45538571329bd5c9b53c (127.21.202.124:46083)
I20260806 19:42:53.184641 29941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.184182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:45842)
0806 19:42:53.184363 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:53.184497 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 19:42:53.184501 29217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfedfa797c6b45538571329bd5c9b53c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8ad649fbf617413fa2b842dc829219f9"
I20260806 19:42:53.185211 29217 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cfedfa797c6b45538571329bd5c9b53c in current term 2: Already voted for candidate 8ad649fbf617413fa2b842dc829219f9 in this term.
I20260806 19:42:53.185174 29410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8ad649fbf617413fa2b842dc829219f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfedfa797c6b45538571329bd5c9b53c"
I20260806 19:42:53.185483 29410 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8ad649fbf617413fa2b842dc829219f9 in current term 2: Already voted for candidate cfedfa797c6b45538571329bd5c9b53c in this term.
I20260806 19:42:53.186192 29941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.185840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:45868)
0806 19:42:53.185989 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:53.186074 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 19:42:53.186213 29945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.185638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:45852)
0806 19:42:53.186002 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:53.186088 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258}
W20260806 19:42:53.186354 29386 leader_election.cc:334] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ec97e781650a49da8405e8eff3b5242e (127.21.202.126:45677): Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
I20260806 19:42:53.186512 29386 leader_election.cc:302] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfedfa797c6b45538571329bd5c9b53c; no voters: 8ad649fbf617413fa2b842dc829219f9, ec97e781650a49da8405e8eff3b5242e
I20260806 19:42:53.186791 29938 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 19:42:53.187279 29945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.186875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:45876)
0806 19:42:53.187091 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:53.187172 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260806 19:42:53.187465 29194 leader_election.cc:334] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ec97e781650a49da8405e8eff3b5242e (127.21.202.126:45677): Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
I20260806 19:42:53.187597 29194 leader_election.cc:302] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ad649fbf617413fa2b842dc829219f9; no voters: cfedfa797c6b45538571329bd5c9b53c, ec97e781650a49da8405e8eff3b5242e
I20260806 19:42:53.187876 29939 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 19:42:53.641952 29948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.641054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.67:58223)
0806 19:42:53.641660 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:53.641749 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":356,"threads_started":1}
I20260806 19:42:53.654969 29949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.654060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.66:60821)
0806 19:42:53.654694 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:53.654789 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":350,"threads_started":1}
I20260806 19:42:53.666141 29950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:53.665291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.65:52941)
0806 19:42:53.665828 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:53.665935 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":268,"threads_started":1}
I20260806 19:42:54.643834 29951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.642982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.67:33339)
0806 19:42:54.643578 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.643667 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":366,"threads_started":1}
I20260806 19:42:54.656730 29952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.655873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.66:56269)
0806 19:42:54.656456 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.656553 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":291,"threads_started":1}
I20260806 19:42:54.667778 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.667081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.65:43915)
0806 19:42:54.667523 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.667606 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":239,"threads_started":1}
I20260806 19:42:54.787627 29954 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:54.787972 29954 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } }
I20260806 19:42:54.788683 29954 leader_election.cc:288] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ec97e781650a49da8405e8eff3b5242e (127.21.202.126:45677), 8ad649fbf617413fa2b842dc829219f9 (127.21.202.125:44803)
I20260806 19:42:54.789346 29217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfedfa797c6b45538571329bd5c9b53c" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8ad649fbf617413fa2b842dc829219f9" is_pre_election: true
I20260806 19:42:54.789646 29217 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfedfa797c6b45538571329bd5c9b53c in term 2.
I20260806 19:42:54.789945 29955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.788998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:46144)
0806 19:42:54.789600 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.789718 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":313,"threads_started":1}
I20260806 19:42:54.790201 29385 leader_election.cc:302] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ad649fbf617413fa2b842dc829219f9, cfedfa797c6b45538571329bd5c9b53c; no voters:
I20260806 19:42:54.790640 29954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 19:42:54.790830 29954 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:54.790958 29954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 2 FOLLOWER]: Advancing to term 3
I20260806 19:42:54.791535 29955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.791156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:46152)
0806 19:42:54.791329 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.791419 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 19:42:54.791725 29386 leader_election.cc:334] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ec97e781650a49da8405e8eff3b5242e (127.21.202.126:45677): Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
I20260806 19:42:54.793599 29954 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } }
I20260806 19:42:54.794265 29954 leader_election.cc:288] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [CANDIDATE]: Term 3 election: Requested vote from peers ec97e781650a49da8405e8eff3b5242e (127.21.202.126:45677), 8ad649fbf617413fa2b842dc829219f9 (127.21.202.125:44803)
I20260806 19:42:54.794844 29955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.794438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:46160)
0806 19:42:54.794579 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.794673 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 19:42:54.794850 29217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfedfa797c6b45538571329bd5c9b53c" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8ad649fbf617413fa2b842dc829219f9"
I20260806 19:42:54.795154 29217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 2 FOLLOWER]: Advancing to term 3
I20260806 19:42:54.795919 29955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.795583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:46162)
0806 19:42:54.795717 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.795796 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 19:42:54.796113 29386 leader_election.cc:334] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer ec97e781650a49da8405e8eff3b5242e (127.21.202.126:45677): Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
I20260806 19:42:54.797806 29217 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfedfa797c6b45538571329bd5c9b53c in term 3.
I20260806 19:42:54.798237 29385 leader_election.cc:302] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ad649fbf617413fa2b842dc829219f9, cfedfa797c6b45538571329bd5c9b53c; no voters: ec97e781650a49da8405e8eff3b5242e
I20260806 19:42:54.798556 29954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 3 FOLLOWER]: Leader election won for term 3
I20260806 19:42:54.799001 29954 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [term 3 LEADER]: Becoming Leader. State: Replica: cfedfa797c6b45538571329bd5c9b53c, State: Running, Role: LEADER
I20260806 19:42:54.799391 29954 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } }
I20260806 19:42:54.803017 29957 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfedfa797c6b45538571329bd5c9b53c. Latest consensus state: current_term: 3 leader_uuid: "cfedfa797c6b45538571329bd5c9b53c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } } }
I20260806 19:42:54.803361 29957 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:54.804104 29959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:54.808065 29959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:54.808908 29959 catalog_manager.cc:1259] Loaded cluster ID: 01055745776b4fecaad51f018b19e9d3
I20260806 19:42:54.808990 29959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:54.810103 29959 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:54.810861 29959 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: Loaded TSK: 0
I20260806 19:42:54.811708 29959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:42:54.890069 29400 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:42:54.897672 29400 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:42:54.902132 29955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.901662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:46200)
0806 19:42:54.901882 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.901975 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 19:42:54.902125 29217 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [term 3 FOLLOWER]: Refusing update from remote peer cfedfa797c6b45538571329bd5c9b53c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 19:42:54.902977 29957 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 19:42:54.903401 29955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.902912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:46214)
0806 19:42:54.903125 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.903270 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 19:42:54.903622 29386 consensus_peers.cc:487] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c -> Peer ec97e781650a49da8405e8eff3b5242e (127.21.202.126:45677): Couldn't send request to peer ec97e781650a49da8405e8eff3b5242e. Status: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 19:42:54.906163 29961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfedfa797c6b45538571329bd5c9b53c. Latest consensus state: current_term: 3 leader_uuid: "cfedfa797c6b45538571329bd5c9b53c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } } }
I20260806 19:42:54.906503 29961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:42:54.908057 29954 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "cfedfa797c6b45538571329bd5c9b53c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } } }
I20260806 19:42:54.908407 29954 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:54.910367 29961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "cfedfa797c6b45538571329bd5c9b53c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec97e781650a49da8405e8eff3b5242e" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 45677 } } peers { permanent_uuid: "8ad649fbf617413fa2b842dc829219f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 44803 } attrs { promote: false } } peers { permanent_uuid: "cfedfa797c6b45538571329bd5c9b53c" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 46083 } attrs { promote: false } } }
I20260806 19:42:54.910805 29961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad649fbf617413fa2b842dc829219f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:42:54.942266 29968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.918612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.21.202.65:36007)
0806 19:42:54.934424 (+ 15812us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:54.934434 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:54.935127 (+ 693us) server_negotiation.cc:407] Connection header received
0806 19:42:54.935436 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:54.935442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:54.935530 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:54.935674 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:54.936532 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:54.937586 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:54.939924 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:54.941125 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:54.941278 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:54.941396 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:54.941746 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 19:42:54.941900 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15617,"thread_start_us":15470,"threads_started":1}
I20260806 19:42:54.942353 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.918474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:36007 (local address 127.0.0.1:43286)
0806 19:42:54.934971 (+ 16497us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.934997 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:54.935179 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:54.935701 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:54.935706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:54.935736 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:54.936340 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:54.936350 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:54.937773 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:54.937783 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:54.939689 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:54.939708 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:54.941320 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:54.941327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:54.941515 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:54.941565 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 19:42:54.941706 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 19:42:54.941870 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16338,"mutex_wait_us":24,"thread_start_us":19861,"threads_started":1}
I20260806 19:42:54.943379 29514 tablet_service.cc:1467] Processing CreateTablet for tablet fd16ffc0bb624276ae50326017d374b3 (DEFAULT_TABLE table=test-table [id=c13097f2d3e047b89394e489b0572a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:54.943888 29514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd16ffc0bb624276ae50326017d374b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:54.944744 29955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.917906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:33619 (local address 127.0.0.1:42878)
0806 19:42:54.918078 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.918098 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:54.918210 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:54.937982 (+ 19772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:54.937987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:54.938020 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:54.938386 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:54.938402 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:54.939918 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:54.939929 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:54.942051 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:54.942065 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:54.943985 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:54.943992 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:54.944175 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:54.944199 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 19:42:54.944306 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 19:42:54.944445 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":17}
I20260806 19:42:54.944850 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.919005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.21.202.66:33619)
0806 19:42:54.937670 (+ 18665us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:54.937679 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:54.937698 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:42:54.937777 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:54.937782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:54.937841 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:54.937962 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:54.938611 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:54.939742 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:54.942215 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:54.943793 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:54.943940 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:54.944043 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:54.944334 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 19:42:54.944489 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18525,"thread_start_us":18413,"threads_started":1}
I20260806 19:42:54.945829 29657 tablet_service.cc:1467] Processing CreateTablet for tablet fd16ffc0bb624276ae50326017d374b3 (DEFAULT_TABLE table=test-table [id=c13097f2d3e047b89394e489b0572a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:54.946280 29657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd16ffc0bb624276ae50326017d374b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:54.946585 29967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.919063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:42981 (local address 127.0.0.1:55930)
0806 19:42:54.938625 (+ 19562us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.938649 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:54.938840 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:54.939413 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:54.939418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:54.939443 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:54.940005 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:54.940028 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:54.941505 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:54.941510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:54.943819 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:54.943834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:54.945648 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:54.945654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:54.945847 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:54.945878 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 19:42:54.945973 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 19:42:54.946102 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19424,"mutex_wait_us":23,"thread_start_us":15563,"threads_started":1}
I20260806 19:42:54.947043 29971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.919505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55930 (local address 127.21.202.67:42981)
0806 19:42:54.937293 (+ 17788us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:54.937299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:54.938733 (+ 1434us) server_negotiation.cc:407] Connection header received
0806 19:42:54.939016 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:54.939022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:54.939097 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:54.939163 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:54.940241 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:54.941387 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:54.944024 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:54.945259 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:54.945359 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:54.945460 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:54.946095 (+ 635us) server_negotiation.cc:299] Negotiation successful
0806 19:42:54.946248 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17647,"thread_start_us":17519,"threads_started":1}
I20260806 19:42:54.949745 29799 tablet_service.cc:1467] Processing CreateTablet for tablet fd16ffc0bb624276ae50326017d374b3 (DEFAULT_TABLE table=test-table [id=c13097f2d3e047b89394e489b0572a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:54.950335 29799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd16ffc0bb624276ae50326017d374b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:54.951015 29972 tablet_bootstrap.cc:492] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c: Bootstrap starting.
I20260806 19:42:54.952869 29972 tablet_bootstrap.cc:654] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:54.952903 29973 tablet_bootstrap.cc:492] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea: Bootstrap starting.
I20260806 19:42:54.954836 29973 tablet_bootstrap.cc:654] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:54.955314 29972 tablet_bootstrap.cc:492] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c: No bootstrap required, opened a new log
I20260806 19:42:54.955775 29972 ts_tablet_manager.cc:1397] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 19:42:54.957600 29972 raft_consensus.cc:348] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } } peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } }
I20260806 19:42:54.957947 29972 raft_consensus.cc:374] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:54.958067 29972 raft_consensus.cc:729] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64104c8ac3b4b4fb0f11c7d035d575c, State: Initialized, Role: FOLLOWER
I20260806 19:42:54.958420 29972 consensus_queue.cc:260] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } } peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } }
I20260806 19:42:54.959632 29972 ts_tablet_manager.cc:1428] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 19:42:54.961410 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.960669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.65:42681)
0806 19:42:54.961021 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.961173 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192}
I20260806 19:42:54.964473 29581 heartbeater.cc:503] Master 127.21.202.124:46083 was elected leader, sending a full tablet report...
I20260806 19:42:54.964949 29973 tablet_bootstrap.cc:492] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea: No bootstrap required, opened a new log
I20260806 19:42:54.965130 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.964594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.65:57591)
0806 19:42:54.964854 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.964951 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260806 19:42:54.965205 29973 ts_tablet_manager.cc:1397] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20260806 19:42:54.966179 29976 tablet_bootstrap.cc:492] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf: Bootstrap starting.
I20260806 19:42:54.968436 29976 tablet_bootstrap.cc:654] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:54.968951 29973 raft_consensus.cc:348] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } } peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } }
I20260806 19:42:54.969256 29973 raft_consensus.cc:374] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:54.969375 29973 raft_consensus.cc:729] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bffa83f0eb452ab48524820f9e1aea, State: Initialized, Role: FOLLOWER
I20260806 19:42:54.969683 29973 consensus_queue.cc:260] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } } peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } }
I20260806 19:42:54.971113 29976 tablet_bootstrap.cc:492] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf: No bootstrap required, opened a new log
I20260806 19:42:54.971300 29976 ts_tablet_manager.cc:1397] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260806 19:42:54.972371 29976 raft_consensus.cc:348] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } } peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } }
I20260806 19:42:54.972692 29976 raft_consensus.cc:374] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:54.972844 29976 raft_consensus.cc:729] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a5049e41f514390804e74e692bc90bf, State: Initialized, Role: FOLLOWER
I20260806 19:42:54.973157 29976 consensus_queue.cc:260] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } } peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } }
I20260806 19:42:54.974913 29951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.974448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.67:46223)
0806 19:42:54.974663 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.974757 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 19:42:54.976678 29976 ts_tablet_manager.cc:1428] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260806 19:42:54.977059 29864 heartbeater.cc:503] Master 127.21.202.124:46083 was elected leader, sending a full tablet report...
I20260806 19:42:54.978181 29951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.977642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.67:42425)
0806 19:42:54.977909 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.978006 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260806 19:42:54.978858 29973 ts_tablet_manager.cc:1428] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea: Time spent starting tablet: real 0.014s user 0.004s sys 0.005s
I20260806 19:42:54.978888 29952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.978450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.66:32999)
0806 19:42:54.978624 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.978717 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 19:42:54.979996 29722 heartbeater.cc:503] Master 127.21.202.124:46083 was elected leader, sending a full tablet report...
I20260806 19:42:54.981338 29952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:54.980836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.66:46915)
0806 19:42:54.981073 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:54.981162 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260806 19:42:55.001371 29977 raft_consensus.cc:482] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:55.001636 29977 raft_consensus.cc:504] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } } peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } }
I20260806 19:42:55.003160 29977 leader_election.cc:288] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3bffa83f0eb452ab48524820f9e1aea (127.21.202.66:33619), 0a5049e41f514390804e74e692bc90bf (127.21.202.67:42981)
I20260806 19:42:55.009097 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.003479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:33619 (local address 127.21.202.65:37619)
0806 19:42:55.003653 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:55.003692 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:55.003841 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:55.004361 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:55.004365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:55.004390 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:55.004651 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:55.004682 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.005688 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.005697 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:55.007099 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:55.007112 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.008235 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.008241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:55.008415 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:55.008439 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 19:42:55.008546 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 19:42:55.008692 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":3}
I20260806 19:42:55.009574 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.003617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:37619 (local address 127.21.202.66:33619)
0806 19:42:55.003830 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:55.003837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:55.003860 (+ 23us) server_negotiation.cc:407] Connection header received
0806 19:42:55.004000 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:55.004005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:55.004065 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:55.004181 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:55.004836 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.005557 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.007259 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.008043 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.008234 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:55.008313 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:55.008574 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 19:42:55.008705 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 19:42:55.009965 29971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.004041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:38501 (local address 127.21.202.67:42981)
0806 19:42:55.004272 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:55.004278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:55.004701 (+ 423us) server_negotiation.cc:407] Connection header received
0806 19:42:55.004902 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:55.004908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:55.004974 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:55.005072 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:55.005572 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.006243 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.007989 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.008749 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.009006 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:55.009090 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:55.009555 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 19:42:55.009694 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 19:42:55.010241 29677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd16ffc0bb624276ae50326017d374b3" candidate_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bffa83f0eb452ab48524820f9e1aea" is_pre_election: true
I20260806 19:42:55.010556 29677 raft_consensus.cc:2454] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d64104c8ac3b4b4fb0f11c7d035d575c in term 0.
I20260806 19:42:55.010784 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.003894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:42981 (local address 127.21.202.65:38501)
0806 19:42:55.004595 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:55.004609 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:55.004720 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:55.005087 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:55.005091 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:55.005114 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:55.005418 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:55.005426 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.006378 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.006385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:55.007820 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:55.007832 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.009083 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.009089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:55.009371 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:55.009404 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 19:42:55.009528 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 19:42:55.009658 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":24,"thread_start_us":340,"threads_started":1}
I20260806 19:42:55.011153 29469 leader_election.cc:302] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3bffa83f0eb452ab48524820f9e1aea, d64104c8ac3b4b4fb0f11c7d035d575c; no voters:
I20260806 19:42:55.011634 29977 raft_consensus.cc:2781] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:42:55.011758 29977 raft_consensus.cc:482] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:55.011873 29977 raft_consensus.cc:3037] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:55.012216 29819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd16ffc0bb624276ae50326017d374b3" candidate_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a5049e41f514390804e74e692bc90bf" is_pre_election: true
I20260806 19:42:55.012633 29819 raft_consensus.cc:2454] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d64104c8ac3b4b4fb0f11c7d035d575c in term 0.
I20260806 19:42:55.015182 29977 raft_consensus.cc:504] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } } peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } }
I20260806 19:42:55.015879 29977 leader_election.cc:288] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [CANDIDATE]: Term 1 election: Requested vote from peers d3bffa83f0eb452ab48524820f9e1aea (127.21.202.66:33619), 0a5049e41f514390804e74e692bc90bf (127.21.202.67:42981)
I20260806 19:42:55.016461 29677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd16ffc0bb624276ae50326017d374b3" candidate_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bffa83f0eb452ab48524820f9e1aea"
I20260806 19:42:55.016505 29819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd16ffc0bb624276ae50326017d374b3" candidate_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a5049e41f514390804e74e692bc90bf"
I20260806 19:42:55.016700 29819 raft_consensus.cc:3037] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:55.016700 29677 raft_consensus.cc:3037] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:55.019865 29677 raft_consensus.cc:2454] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d64104c8ac3b4b4fb0f11c7d035d575c in term 1.
I20260806 19:42:55.019871 29819 raft_consensus.cc:2454] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d64104c8ac3b4b4fb0f11c7d035d575c in term 1.
I20260806 19:42:55.020344 29469 leader_election.cc:302] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3bffa83f0eb452ab48524820f9e1aea, d64104c8ac3b4b4fb0f11c7d035d575c; no voters:
I20260806 19:42:55.020689 29977 raft_consensus.cc:2781] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:55.021236 29977 raft_consensus.cc:686] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [term 1 LEADER]: Becoming Leader. State: Replica: d64104c8ac3b4b4fb0f11c7d035d575c, State: Running, Role: LEADER
I20260806 19:42:55.021665 29977 consensus_queue.cc:237] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } } peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } }
I20260806 19:42:55.022686 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.022223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.65:46049)
0806 19:42:55.022399 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:55.022522 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 19:42:55.025002 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.024529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.65:44787)
0806 19:42:55.024707 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:55.024834 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 19:42:55.025800 29400 catalog_manager.cc:5495] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c reported cstate change: term changed from 0 to 1, leader changed from <none> to d64104c8ac3b4b4fb0f11c7d035d575c (127.21.202.65). New cstate: current_term: 1 leader_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d64104c8ac3b4b4fb0f11c7d035d575c" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 } health_report { overall_health: UNKNOWN } } }
I20260806 19:42:55.055109 29968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.049488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43300 (local address 127.21.202.65:36007)
0806 19:42:55.049756 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:55.049763 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:55.049790 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:42:55.049883 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:55.049887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:55.049971 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:55.050131 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:55.050534 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.051352 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.052794 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.053245 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.053479 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:55.053927 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:55.054485 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:55.054647 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:55.054729 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 19:42:55.054829 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 19:42:55.059648 29677 raft_consensus.cc:1261] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea [term 1 FOLLOWER]: Refusing update from remote peer d64104c8ac3b4b4fb0f11c7d035d575c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:55.059947 29819 raft_consensus.cc:1261] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf [term 1 FOLLOWER]: Refusing update from remote peer d64104c8ac3b4b4fb0f11c7d035d575c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:55.060462 29982 consensus_queue.cc:1035] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bffa83f0eb452ab48524820f9e1aea" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:55.060968 29977 consensus_queue.cc:1035] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a5049e41f514390804e74e692bc90bf" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:55.063463 29952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.062884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.66:42743)
0806 19:42:55.063136 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:55.063252 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260806 19:42:55.064090 29951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.063445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.67:37597)
0806 19:42:55.063743 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:55.063884 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260806 19:42:55.070618 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.070143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.65:57857)
0806 19:42:55.070336 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:55.070436 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 19:42:55.070966 29952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.070210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.66:45023)
0806 19:42:55.070605 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:55.070723 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237}
I20260806 19:42:55.083118 29400 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:55.083469 29400 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:42:55.083689 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.083133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.65:56667)
0806 19:42:55.083352 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:55.083450 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 19:42:55.086606 29951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.085896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.21.202.67:45431)
0806 19:42:55.086207 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:55.086327 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260806 19:42:55.092031 29400 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfedfa797c6b45538571329bd5c9b53c: Sending DeleteTablet for 3 replicas of tablet fd16ffc0bb624276ae50326017d374b3
I20260806 19:42:55.093206 29657 tablet_service.cc:1514] Processing DeleteTablet for tablet fd16ffc0bb624276ae50326017d374b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:55 UTC) from {username='oryx', principal='oryx/127.21.202.124@KRBTEST.COM'} at 127.0.0.1:42878
I20260806 19:42:55.094034 29514 tablet_service.cc:1514] Processing DeleteTablet for tablet fd16ffc0bb624276ae50326017d374b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:55 UTC) from {username='oryx', principal='oryx/127.21.202.124@KRBTEST.COM'} at 127.0.0.1:43286
I20260806 19:42:55.094786 29799 tablet_service.cc:1514] Processing DeleteTablet for tablet fd16ffc0bb624276ae50326017d374b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:55 UTC) from {username='oryx', principal='oryx/127.21.202.124@KRBTEST.COM'} at 127.0.0.1:55930
I20260806 19:42:55.098658 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 29387
I20260806 19:42:55.104568 29994 tablet_replica.cc:330] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea: stopping tablet replica
I20260806 19:42:55.104979 29994 raft_consensus.cc:2229] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:55.105347 29994 raft_consensus.cc:2258] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:55.106019 29996 tablet_replica.cc:330] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf: stopping tablet replica
I20260806 19:42:55.106279 29996 raft_consensus.cc:2229] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:42:55.106666 29996 raft_consensus.cc:2258] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:55.107584 29995 tablet_replica.cc:330] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c: stopping tablet replica
I20260806 19:42:55.107761 29994 ts_tablet_manager.cc:1905] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:55.107808 29995 raft_consensus.cc:2229] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:42:55.108407 29995 raft_consensus.cc:2258] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:42:55.110052 29995 ts_tablet_manager.cc:1905] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:55.110700 29996 ts_tablet_manager.cc:1905] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:42:55.114960 29994 ts_tablet_manager.cc:1918] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:55.115118 29994 log.cc:1196] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/fd16ffc0bb624276ae50326017d374b3
I20260806 19:42:55.115588 29994 ts_tablet_manager.cc:1939] T fd16ffc0bb624276ae50326017d374b3 P d3bffa83f0eb452ab48524820f9e1aea: Deleting consensus metadata
I20260806 19:42:55.116897 29385 catalog_manager.cc:4841] TS d3bffa83f0eb452ab48524820f9e1aea (127.21.202.66:33619): tablet fd16ffc0bb624276ae50326017d374b3 (table test-table [id=c13097f2d3e047b89394e489b0572a25]) successfully deleted
I20260806 19:42:55.117271 29996 ts_tablet_manager.cc:1918] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:55.117442 29996 log.cc:1196] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/fd16ffc0bb624276ae50326017d374b3
I20260806 19:42:55.117676 29995 ts_tablet_manager.cc:1918] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:42:55.117837 29995 log.cc:1196] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/fd16ffc0bb624276ae50326017d374b3
I20260806 19:42:55.117944 29996 ts_tablet_manager.cc:1939] T fd16ffc0bb624276ae50326017d374b3 P 0a5049e41f514390804e74e692bc90bf: Deleting consensus metadata
I20260806 19:42:55.118345 29995 ts_tablet_manager.cc:1939] T fd16ffc0bb624276ae50326017d374b3 P d64104c8ac3b4b4fb0f11c7d035d575c: Deleting consensus metadata
I20260806 19:42:55.119966 29385 catalog_manager.cc:4841] TS 0a5049e41f514390804e74e692bc90bf (127.21.202.67:42981): tablet fd16ffc0bb624276ae50326017d374b3 (table test-table [id=c13097f2d3e047b89394e489b0572a25]) successfully deleted
I20260806 19:42:55.120244 29386 catalog_manager.cc:4841] TS d64104c8ac3b4b4fb0f11c7d035d575c (127.21.202.65:36007): tablet fd16ffc0bb624276ae50326017d374b3 (table test-table [id=c13097f2d3e047b89394e489b0572a25]) successfully deleted
I20260806 19:42:55.130033 29968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.099998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43310 (local address 127.21.202.65:36007)
0806 19:42:55.100177 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:55.100185 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:55.102671 (+ 2486us) server_negotiation.cc:407] Connection header received
0806 19:42:55.102870 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:55.102876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:55.102969 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:55.103088 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:55.104940 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.105978 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.110085 (+ 4107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.110777 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.110908 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:55.123073 (+ 12165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:55.123086 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:55.123103 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:55.123141 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:55.126637 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:55.127535 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:55.127542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:55.127545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:55.127703 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:55.128164 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:55.128177 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:55.128180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:55.128589 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:55.129003 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:55.129360 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 19:42:55.129628 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 19:42:55.130848 29575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:55.236663 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 29387
I20260806 19:42:55.257961 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 29593
I20260806 19:42:55.273188 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.258857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.21.202.66:33619)
0806 19:42:55.259068 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:55.259076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:55.259169 (+ 93us) server_negotiation.cc:407] Connection header received
0806 19:42:55.259414 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:55.259424 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:55.259535 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:55.259656 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:55.261163 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.261949 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.263176 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.263668 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.263805 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:55.266802 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:55.266819 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:55.266836 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:55.266872 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:55.270243 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:55.270977 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:55.270985 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:55.270987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:55.271128 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:55.271518 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:55.271524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:55.271527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:55.271921 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:55.272254 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:55.272561 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 19:42:55.272856 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 19:42:55.273913 29717 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:55.353622 29967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.353072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:46226)
0806 19:42:55.353347 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:55.353458 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260806 19:42:55.354875 29967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.354455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:45677 (local address 127.0.0.1:46234)
0806 19:42:55.354624 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:55.354719 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 19:42:55.385614 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 29593
I20260806 19:42:55.404940 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 29736
I20260806 19:42:55.419731 29971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:55.405857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55932 (local address 127.21.202.67:42981)
0806 19:42:55.406146 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:55.406152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:55.406322 (+ 170us) server_negotiation.cc:407] Connection header received
0806 19:42:55.406540 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:55.406544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:55.406614 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:55.406709 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:55.408304 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.409460 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.410719 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:55.411123 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:55.411303 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:55.414024 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:55.414041 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:55.414058 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:55.414094 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:55.416919 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:55.417608 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:55.417617 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:55.417621 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:55.417755 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:55.418147 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:55.418153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:55.418156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:55.418529 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:55.418862 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:55.419158 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 19:42:55.419400 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 19:42:55.420508 29859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:55.547237 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 29736
I20260806 19:42:55.573150 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 29063
I20260806 19:42:55.574501 29237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:55.697682 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 29063
I20260806 19:42:55.725484 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 29163
I20260806 19:42:55.726744 29430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:42:55.858491 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 29163
2026-08-06T19:42:55Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11776 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:42:55 dist-test-slave-77v9 krb5kdc[30005](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:42:55 dist-test-slave-77v9 krb5kdc[30005](info): set up 2 sockets
Aug 06 19:42:55 dist-test-slave-77v9 krb5kdc[30005](info): commencing operation
krb5kdc: starting...
W20260806 19:42:57.011427 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:42:57 dist-test-slave-77v9 krb5kdc[30005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:42:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:42:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.126.keytab.
Entry for principal oryx/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.126.keytab.
I20260806 19:42:57.273408 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46343
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.126.keytab
--principal=oryx/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:46343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:57.622108 30019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:57.622493 30019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:57.622677 30019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:57.630373 30019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:42:57.630542 30019 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 19:42:57.630652 30019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:57.630781 30019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:57.630884 30019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:42:57.630998 30019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:42:57.641516 30019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:46343
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.126.keytab
--principal=oryx/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:57.642980 30019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:57.644404 30019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:57.658133 30025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:57.658193 30027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:57.658278 30024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:57.658785 30019 server_base.cc:1034] running on GCE node
I20260806 19:42:57.659754 30019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:57.661312 30019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:57.662631 30019 hybrid_clock.cc:648] HybridClock initialized: now 1786045377662545 us; error 35 us; skew 500 ppm
I20260806 19:42:57.663043 30019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:57 dist-test-slave-77v9 krb5kdc[30005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045377, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:57.666695 30019 init.cc:377] Logged in from keytab as oryx/127.21.202.126@KRBTEST.COM (short username oryx)
I20260806 19:42:57.668428 30019 webserver.cc:460] Webserver started at http://127.21.202.126:42787/ using document root <none> and password file <none>
I20260806 19:42:57.669318 30019 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:57.669468 30019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:57.670024 30019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:57.673825 30019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "63bdaa10658146dab4b821f4dcb52f61"
format_stamp: "Formatted at 2026-08-06 19:42:57 on dist-test-slave-77v9"
server_key: "0e5f91388c993a0a1329be68cd3f7f30"
server_key_iv: "3eff2d4b3f5f74813ebe997300f11baa"
server_key_version: "encryptionkey@0"
I20260806 19:42:57.674993 30019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "63bdaa10658146dab4b821f4dcb52f61"
format_stamp: "Formatted at 2026-08-06 19:42:57 on dist-test-slave-77v9"
server_key: "0e5f91388c993a0a1329be68cd3f7f30"
server_key_iv: "3eff2d4b3f5f74813ebe997300f11baa"
server_key_version: "encryptionkey@0"
I20260806 19:42:57.681041 30019 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 19:42:57.685001 30034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:57.686444 30019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:42:57.686661 30019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "63bdaa10658146dab4b821f4dcb52f61"
format_stamp: "Formatted at 2026-08-06 19:42:57 on dist-test-slave-77v9"
server_key: "0e5f91388c993a0a1329be68cd3f7f30"
server_key_iv: "3eff2d4b3f5f74813ebe997300f11baa"
server_key_version: "encryptionkey@0"
I20260806 19:42:57.686904 30019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:57.725189 30019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:57.730036 30019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:57.730358 30019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:57.755088 30019 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:46343
I20260806 19:42:57.755098 30095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:46343 every 8 connection(s)
I20260806 19:42:57.756603 30019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:42:57.760296 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 30019
I20260806 19:42:57.760581 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:42:57.760919 30096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:57.761426 22313 external_mini_cluster.cc:1428] Setting key 2475bb12a6b3102039039442e715551a
I20260806 19:42:57.776515 30096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61: Bootstrap starting.
Aug 06 19:42:57 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
I20260806 19:42:57.780714 30096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:57.781993 30096 log.cc:824] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:57.785597 30096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61: No bootstrap required, opened a new log
I20260806 19:42:57.788208 30099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:57.763507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40150 (local address 127.21.202.126:46343)
0806 19:42:57.764390 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:57.764403 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:57.764449 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:42:57.766287 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:57.766309 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:57.766730 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:57.767091 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:57.769225 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:57.770732 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:57.772775 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:57.773667 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:57.773827 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:57.779043 (+ 5216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:57.779052 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:57.779067 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:57.779110 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:57.782651 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:57.783545 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:57.783554 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:57.783561 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:57.783754 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:57.784189 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:57.784197 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:57.784216 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:57.785203 (+ 987us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:57.785660 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:57.786184 (+ 524us) server_negotiation.cc:299] Negotiation successful
0806 19:42:57.786716 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":438,"threads_started":1}
I20260806 19:42:57.795104 30096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63bdaa10658146dab4b821f4dcb52f61" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46343 } }
I20260806 19:42:57.795459 30096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:57.795536 30096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63bdaa10658146dab4b821f4dcb52f61, State: Initialized, Role: FOLLOWER
I20260806 19:42:57.796104 30096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63bdaa10658146dab4b821f4dcb52f61" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46343 } }
I20260806 19:42:57.796314 30096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:42:57.796411 30096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:42:57.796537 30096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:57.799044 30096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63bdaa10658146dab4b821f4dcb52f61" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46343 } }
I20260806 19:42:57.799489 30096 leader_election.cc:302] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63bdaa10658146dab4b821f4dcb52f61; no voters:
I20260806 19:42:57.800107 30096 leader_election.cc:288] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:42:57.800274 30101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:57.801015 30101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [term 1 LEADER]: Becoming Leader. State: Replica: 63bdaa10658146dab4b821f4dcb52f61, State: Running, Role: LEADER
I20260806 19:42:57.801586 30101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63bdaa10658146dab4b821f4dcb52f61" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46343 } }
I20260806 19:42:57.802385 30096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:42:57.804270 30103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63bdaa10658146dab4b821f4dcb52f61. Latest consensus state: current_term: 1 leader_uuid: "63bdaa10658146dab4b821f4dcb52f61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63bdaa10658146dab4b821f4dcb52f61" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46343 } } }
I20260806 19:42:57.804643 30103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:57.804425 30102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63bdaa10658146dab4b821f4dcb52f61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63bdaa10658146dab4b821f4dcb52f61" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46343 } } }
I20260806 19:42:57.805222 30102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61 [sys.catalog]: This master's current role is: LEADER
I20260806 19:42:57.806934 30107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:42:57.812795 30107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:42:57.823781 30107 catalog_manager.cc:1347] Generated new cluster ID: 4468555215af440790aab569f1fe8b92
I20260806 19:42:57.823935 30107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:42:57.837638 30107 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:42:57.839107 30107 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:42:57.852484 30107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61: Generated new TSK 0
I20260806 19:42:57.853406 30107 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab.
Entry for principal oryx/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab.
I20260806 19:42:57.921726 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46343
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab
--principal=oryx/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:58.275727 30122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:58.276084 30122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:58.276276 30122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:58.283669 30122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:58.283828 30122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:58.283922 30122 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:42:58.284005 30122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:42:58.294376 30122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab
--principal=oryx/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.202.126:46343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:58.295634 30122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:58.297026 30122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:58.310739 30130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:58.310789 30128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:58.310829 30127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:58.311906 30122 server_base.cc:1034] running on GCE node
I20260806 19:42:58.312477 30122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:58.313498 30122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:58.314735 30122 hybrid_clock.cc:648] HybridClock initialized: now 1786045378314639 us; error 61 us; skew 500 ppm
I20260806 19:42:58.315061 30122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:58 dist-test-slave-77v9 krb5kdc[30005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045378, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:58.318331 30122 init.cc:377] Logged in from keytab as oryx/127.21.202.65@KRBTEST.COM (short username oryx)
I20260806 19:42:58.319790 30122 webserver.cc:460] Webserver started at http://127.21.202.65:34169/ using document root <none> and password file <none>
I20260806 19:42:58.320495 30122 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:58.320581 30122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:58.320925 30122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:58.323861 30122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "cc7f144fed494853a610eac95b529149"
format_stamp: "Formatted at 2026-08-06 19:42:58 on dist-test-slave-77v9"
server_key: "4752bf879703d5afd25be11642c48c51"
server_key_iv: "b4795cc6db7e894ddc63adb2ad46d05b"
server_key_version: "encryptionkey@0"
I20260806 19:42:58.324697 30122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "cc7f144fed494853a610eac95b529149"
format_stamp: "Formatted at 2026-08-06 19:42:58 on dist-test-slave-77v9"
server_key: "4752bf879703d5afd25be11642c48c51"
server_key_iv: "b4795cc6db7e894ddc63adb2ad46d05b"
server_key_version: "encryptionkey@0"
I20260806 19:42:58.330279 30122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 19:42:58.333856 30137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:58.335295 30122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 19:42:58.335489 30122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "cc7f144fed494853a610eac95b529149"
format_stamp: "Formatted at 2026-08-06 19:42:58 on dist-test-slave-77v9"
server_key: "4752bf879703d5afd25be11642c48c51"
server_key_iv: "b4795cc6db7e894ddc63adb2ad46d05b"
server_key_version: "encryptionkey@0"
I20260806 19:42:58.335760 30122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:58.349290 30122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:58.354231 30122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:58.354497 30122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:58.357414 30122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:58.357543 30122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:58.357682 30122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:58.357769 30122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:42:58 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045378, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
I20260806 19:42:58.385392 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:58.359063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46343 (local address 127.21.202.65:36335)
0806 19:42:58.360051 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:58.360100 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:58.362335 (+ 2235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:58.363500 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:58.363519 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:58.364851 (+ 1332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:58.366086 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:58.366126 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.367577 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.367581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:58.369113 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:58.369147 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.370637 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.370649 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:58.370876 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:58.371750 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:58.371774 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:58.374771 (+ 2997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:58.379470 (+ 4699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:58.379483 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:58.379514 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:58.380178 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:58.380675 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:58.380681 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:58.380685 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:58.381068 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:58.381891 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:58.381907 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:58.382473 (+ 566us) client_negotiation.cc:769] Sending connection context
0806 19:42:58.382938 (+ 465us) client_negotiation.cc:240] Negotiation successful
0806 19:42:58.383634 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":436,"threads_started":1}
I20260806 19:42:58.385900 30147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:58.358114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:36335 (local address 127.21.202.126:46343)
0806 19:42:58.358819 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:58.358828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:58.360216 (+ 1388us) server_negotiation.cc:407] Connection header received
0806 19:42:58.362680 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:58.362688 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:58.362809 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:58.362973 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:58.366370 (+ 3397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.367394 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.369323 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.370336 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.370542 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:58.375074 (+ 4532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:58.375084 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:58.375091 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:58.375140 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:58.379277 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:58.380399 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:58.380405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:58.380407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:58.380541 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:58.381237 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:58.381245 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:58.381249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:58.381707 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:58.381885 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:58.385045 (+ 3160us) server_negotiation.cc:299] Negotiation successful
0806 19:42:58.385377 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":442,"threads_started":1}
I20260806 19:42:58.407481 30122 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:37773
I20260806 19:42:58.407508 30258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:37773 every 8 connection(s)
I20260806 19:42:58.408890 30122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:42:58.410226 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 30122
I20260806 19:42:58.410487 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:42:58.411396 22313 external_mini_cluster.cc:1428] Setting key 6d7895adbd29ff85f871cb3c68eea67b
I20260806 19:42:58.412321 30229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:58.394176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46343 (local address 127.21.202.65:38529)
0806 19:42:58.394855 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:58.394875 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:58.395083 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:58.395452 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:58.395460 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:58.396283 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:58.396955 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:58.396976 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.398559 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.398569 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:58.400678 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:58.400704 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.402068 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.402080 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:58.402342 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:58.403210 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:58.403232 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:58.404530 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:58.408628 (+ 4098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:58.408634 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:58.408638 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:58.409236 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:58.409673 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:58.409681 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:58.409685 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:58.409878 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:58.410783 (+ 905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:58.410789 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:58.411015 (+ 226us) client_negotiation.cc:769] Sending connection context
0806 19:42:58.411206 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 19:42:58.411626 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":378,"threads_started":1}
I20260806 19:42:58.412546 30147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:58.394037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:38529 (local address 127.21.202.126:46343)
0806 19:42:58.394205 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:58.394208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:58.394984 (+ 776us) server_negotiation.cc:407] Connection header received
0806 19:42:58.395225 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:58.395231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:58.395313 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:58.395420 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:58.397171 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.398328 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.400994 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.401731 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.401904 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:58.404869 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:58.404878 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:58.404886 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:58.404937 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:58.408470 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:58.409363 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:58.409371 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:58.409375 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:58.409521 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:58.410048 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:58.410053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:58.410059 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:58.410506 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:58.410703 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:58.411882 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0806 19:42:58.412144 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 19:42:58.411669 30259 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46343
I20260806 19:42:58.413257 30259 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:58.414244 30259 heartbeater.cc:511] Master 127.21.202.126:46343 requested a full tablet report, sending...
I20260806 19:42:58.417493 30051 ts_manager.cc:194] Registered new tserver with Master: cc7f144fed494853a610eac95b529149 (127.21.202.65:37773)
I20260806 19:42:58.419747 30051 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:36335
WARNING: no policy specified for oryx/127.21.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab.
Entry for principal oryx/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab.
I20260806 19:42:58.473348 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46343
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab
--principal=oryx/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:58.816946 30263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:58.817282 30263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:58.817384 30263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:58.826068 30263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:58.826187 30263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:58.826272 30263 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:42:58.826354 30263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:42:58.836283 30263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab
--principal=oryx/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.202.126:46343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:58.837654 30263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:58.839012 30263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:58.852551 30271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:58.852592 30269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:58.853065 30268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:58.853520 30263 server_base.cc:1034] running on GCE node
I20260806 19:42:58.854223 30263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:58.855118 30263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:58.856365 30263 hybrid_clock.cc:648] HybridClock initialized: now 1786045378856308 us; error 34 us; skew 500 ppm
I20260806 19:42:58.856657 30263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:42:58 dist-test-slave-77v9 krb5kdc[30005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045378, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:58.860123 30263 init.cc:377] Logged in from keytab as oryx/127.21.202.66@KRBTEST.COM (short username oryx)
I20260806 19:42:58.861812 30263 webserver.cc:460] Webserver started at http://127.21.202.66:46669/ using document root <none> and password file <none>
I20260806 19:42:58.862591 30263 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:58.862735 30263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:58.863081 30263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:58.866164 30263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "495b6134be8e4b7db72d06fe0900920e"
format_stamp: "Formatted at 2026-08-06 19:42:58 on dist-test-slave-77v9"
server_key: "f1f8d1b5cae99f5b17d19103265be54f"
server_key_iv: "116f9f201bed92a123c328b2094bfd21"
server_key_version: "encryptionkey@0"
I20260806 19:42:58.867009 30263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "495b6134be8e4b7db72d06fe0900920e"
format_stamp: "Formatted at 2026-08-06 19:42:58 on dist-test-slave-77v9"
server_key: "f1f8d1b5cae99f5b17d19103265be54f"
server_key_iv: "116f9f201bed92a123c328b2094bfd21"
server_key_version: "encryptionkey@0"
I20260806 19:42:58.872372 30263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 19:42:58.876189 30278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:58.877682 30263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 19:42:58.877862 30263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "495b6134be8e4b7db72d06fe0900920e"
format_stamp: "Formatted at 2026-08-06 19:42:58 on dist-test-slave-77v9"
server_key: "f1f8d1b5cae99f5b17d19103265be54f"
server_key_iv: "116f9f201bed92a123c328b2094bfd21"
server_key_version: "encryptionkey@0"
I20260806 19:42:58.878106 30263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:58.896796 30263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:58.901300 30263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:58.901607 30263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:58.904418 30263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:58.904559 30263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:58.904727 30263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:58.904825 30263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:42:58 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045378, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
I20260806 19:42:58.934585 30147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:58.905316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:34239 (local address 127.21.202.126:46343)
0806 19:42:58.905516 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:58.905535 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:58.907245 (+ 1710us) server_negotiation.cc:407] Connection header received
0806 19:42:58.910508 (+ 3263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:58.910517 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:58.910619 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:58.910733 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:58.913298 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.914322 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.918536 (+ 4214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.919487 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.919710 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:58.925126 (+ 5416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:58.925133 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:58.925137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:58.925182 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:58.929519 (+ 4337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:58.930919 (+ 1400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:58.930927 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:58.930932 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:58.931126 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:58.931805 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:58.931812 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:58.931816 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:58.932175 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:58.932387 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:58.933849 (+ 1462us) server_negotiation.cc:299] Negotiation successful
0806 19:42:58.934114 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 19:42:58.936384 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:58.905958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46343 (local address 127.21.202.66:34239)
0806 19:42:58.906898 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:58.907112 (+ 214us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:58.910024 (+ 2912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:58.911178 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:58.911194 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:58.912050 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:58.913026 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:58.913060 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.914543 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.914556 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:58.918280 (+ 3724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:58.918312 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.919836 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.919849 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:58.920140 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:58.921146 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:58.921176 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:58.924711 (+ 3535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:58.929769 (+ 5058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:58.929787 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:58.929807 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:58.930680 (+ 873us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:58.931303 (+ 623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:58.931312 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:58.931322 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:58.931633 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:58.932345 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:58.932357 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:58.932874 (+ 517us) client_negotiation.cc:769] Sending connection context
0806 19:42:58.933296 (+ 422us) client_negotiation.cc:240] Negotiation successful
0806 19:42:58.934103 (+ 807us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":322,"threads_started":1}
I20260806 19:42:58.961638 30147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:58.944546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:58633 (local address 127.21.202.126:46343)
0806 19:42:58.944835 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:58.944844 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:58.945332 (+ 488us) server_negotiation.cc:407] Connection header received
0806 19:42:58.945564 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:58.945573 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:58.945682 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:58.945803 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:58.946778 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.948327 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.950429 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.951041 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.951185 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:58.953844 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:58.953849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:58.953852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:58.953888 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:58.957359 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:58.958687 (+ 1328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:58.958695 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:58.958701 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:58.958882 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:58.959417 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:58.959425 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:58.959430 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:58.960069 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:58.960280 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:58.960730 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 19:42:58.961190 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 19:42:58.961676 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:58.944461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46343 (local address 127.21.202.66:58633)
0806 19:42:58.945150 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:58.945177 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:58.945368 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:58.945863 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:58.945866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:58.946296 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:58.946616 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:58.946626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.948540 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.948545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:58.950174 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:58.950199 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:58.951285 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:58.951293 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:58.951461 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:58.952323 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:58.952341 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:58.953616 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:58.957595 (+ 3979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:58.957605 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:58.957611 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:58.958493 (+ 882us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:58.959025 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:58.959030 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:58.959034 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:58.959256 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:58.960237 (+ 981us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:58.960241 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:58.960417 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 19:42:58.960552 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 19:42:58.960934 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":373,"threads_started":1}
I20260806 19:42:58.965252 30397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:42377 every 8 connection(s)
I20260806 19:42:58.965250 30263 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:42377
I20260806 19:42:58.967053 30263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:42:58.969473 30398 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46343
I20260806 19:42:58.969750 30398 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:58.970165 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 30263
I20260806 19:42:58.970387 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:42:58.970638 30398 heartbeater.cc:511] Master 127.21.202.126:46343 requested a full tablet report, sending...
I20260806 19:42:58.970945 22313 external_mini_cluster.cc:1428] Setting key dbd2fb9fe0c3b5713dfbbb290c71cf65
I20260806 19:42:58.972290 30051 ts_manager.cc:194] Registered new tserver with Master: 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377)
I20260806 19:42:58.973263 30051 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:34239
WARNING: no policy specified for oryx/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab.
Entry for principal oryx/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab.
I20260806 19:42:59.032148 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46343
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab
--principal=oryx/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:42:59.379837 30402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:42:59.380203 30402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:42:59.380383 30402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:42:59.389042 30402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:42:59.389212 30402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:42:59.389360 30402 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:42:59.389501 30402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:42:59.399749 30402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab
--principal=oryx/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.202.126:46343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:42:59.401216 30402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:42:59.402668 30402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:42:59.417969 30407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:59.417820 30410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:42:59.417882 30408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:42:59.418938 30402 server_base.cc:1034] running on GCE node
I20260806 19:42:59.419658 30402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:42:59.420625 30402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:42:59.421944 30402 hybrid_clock.cc:648] HybridClock initialized: now 1786045379421861 us; error 53 us; skew 500 ppm
I20260806 19:42:59.422300 30402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:42:59.422725 30259 heartbeater.cc:503] Master 127.21.202.126:46343 was elected leader, sending a full tablet report...
Aug 06 19:42:59 dist-test-slave-77v9 krb5kdc[30005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045379, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:42:59.426290 30402 init.cc:377] Logged in from keytab as oryx/127.21.202.67@KRBTEST.COM (short username oryx)
I20260806 19:42:59.428541 30402 webserver.cc:460] Webserver started at http://127.21.202.67:42975/ using document root <none> and password file <none>
I20260806 19:42:59.429618 30402 fs_manager.cc:359] Metadata directory not provided
I20260806 19:42:59.429765 30402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:42:59.430187 30402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:42:59.434290 30402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "ec750bd3403b47cba074d1b138e717a3"
format_stamp: "Formatted at 2026-08-06 19:42:59 on dist-test-slave-77v9"
server_key: "7f910ddfd5b5e85752a19849c92d3cb9"
server_key_iv: "a90dd85b61e44ee8e0e02b5a8ef04f76"
server_key_version: "encryptionkey@0"
I20260806 19:42:59.435163 30402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "ec750bd3403b47cba074d1b138e717a3"
format_stamp: "Formatted at 2026-08-06 19:42:59 on dist-test-slave-77v9"
server_key: "7f910ddfd5b5e85752a19849c92d3cb9"
server_key_iv: "a90dd85b61e44ee8e0e02b5a8ef04f76"
server_key_version: "encryptionkey@0"
I20260806 19:42:59.444020 30402 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.005s
I20260806 19:42:59.448630 30417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:59.450254 30402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 19:42:59.450428 30402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "ec750bd3403b47cba074d1b138e717a3"
format_stamp: "Formatted at 2026-08-06 19:42:59 on dist-test-slave-77v9"
server_key: "7f910ddfd5b5e85752a19849c92d3cb9"
server_key_iv: "a90dd85b61e44ee8e0e02b5a8ef04f76"
server_key_version: "encryptionkey@0"
I20260806 19:42:59.450644 30402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:42:59.465337 30402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:42:59.471400 30402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:42:59.471704 30402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:42:59.474591 30402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:42:59.474704 30402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:42:59.474864 30402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:42:59.474946 30402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:42:59 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045379, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
I20260806 19:42:59.507337 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.475578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:52735 (local address 127.21.202.126:46343)
0806 19:42:59.476110 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:59.476117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:59.477885 (+ 1768us) server_negotiation.cc:407] Connection header received
0806 19:42:59.481606 (+ 3721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:59.481618 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:59.481774 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:59.481973 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:59.483780 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.485445 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.488256 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.488990 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.489136 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.495762 (+ 6626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:59.495772 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:59.495779 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:59.495833 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:59.500506 (+ 4673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:59.502181 (+ 1675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:59.502188 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:59.502192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:59.502387 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:59.504277 (+ 1890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:59.504288 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:59.504295 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:59.504961 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:59.505170 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:59.506239 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0806 19:42:59.506625 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":288,"threads_started":1}
I20260806 19:42:59.509409 30427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.476278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46343 (local address 127.21.202.67:52735)
0806 19:42:59.477300 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:59.477599 (+ 299us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:59.481086 (+ 3487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:59.482298 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:59.482319 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:59.482920 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:59.483557 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.483576 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.485673 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.485681 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.488014 (+ 2333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.488080 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.489287 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.489295 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.489586 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.490448 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:59.490491 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:59.495362 (+ 4871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:59.500802 (+ 5440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:59.500847 (+ 45us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:59.500878 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:59.501936 (+ 1058us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:59.502533 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:59.502539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:59.502543 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:59.504071 (+ 1528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:59.505190 (+ 1119us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:59.505202 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:59.505665 (+ 463us) client_negotiation.cc:769] Sending connection context
0806 19:42:59.506032 (+ 367us) client_negotiation.cc:240] Negotiation successful
0806 19:42:59.506849 (+ 817us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":481,"threads_started":1}
I20260806 19:42:59.526489 30402 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:44865
I20260806 19:42:59.526502 30540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:44865 every 8 connection(s)
I20260806 19:42:59.527799 30402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:42:59.530433 30541 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46343
I20260806 19:42:59.530699 30541 heartbeater.cc:464] Registering TS with master...
I20260806 19:42:59.531459 30541 heartbeater.cc:511] Master 127.21.202.126:46343 requested a full tablet report, sending...
I20260806 19:42:59.531641 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 30402
I20260806 19:42:59.531888 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:42:59.532541 22313 external_mini_cluster.cc:1428] Setting key 55bb27f5ff9fc27d788bb263e3071693
I20260806 19:42:59.533228 30051 ts_manager.cc:194] Registered new tserver with Master: ec750bd3403b47cba074d1b138e717a3 (127.21.202.67:44865)
I20260806 19:42:59.534616 30517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.517220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46343 (local address 127.21.202.67:55877)
0806 19:42:59.517939 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:59.517965 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:59.518151 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:59.518536 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:59.518541 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:59.519306 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:42:59.520020 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.520043 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.521846 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.521851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.524128 (+ 2277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.524146 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.525114 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.525121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.525359 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.526167 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:42:59.526186 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:42:59.527132 (+ 946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:42:59.531190 (+ 4058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:59.531195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:59.531199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:59.531894 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:59.532428 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:42:59.532436 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:42:59.532441 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:42:59.532619 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:42:59.533354 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:42:59.533360 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:42:59.533579 (+ 219us) client_negotiation.cc:769] Sending connection context
0806 19:42:59.533751 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 19:42:59.534127 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":400,"threads_started":1}
I20260806 19:42:59.534605 30051 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:52735
I20260806 19:42:59.536216 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.517291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:55877 (local address 127.21.202.126:46343)
0806 19:42:59.517548 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:59.517557 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:59.518072 (+ 515us) server_negotiation.cc:407] Connection header received
0806 19:42:59.518296 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:59.518301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:59.518381 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:59.518500 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:59.520241 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.521597 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.524302 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.524767 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.525048 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.527333 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:59.527344 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:59.527347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:59.527381 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:59.531035 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:59.532073 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:59.532079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:59.532083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:59.532273 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:59.532775 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:59.532778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:59.532781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:59.533156 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:59.533353 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:59.535534 (+ 2181us) server_negotiation.cc:299] Negotiation successful
0806 19:42:59.535830 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 19:42:59.537041 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 19:42:59 dist-test-slave-77v9 krb5kdc[30005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045379, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 19:42:59 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:42:59 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 19:42:59.600345 30426 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:42:59.588686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47722 (local address 127.21.202.126:46343)
0806 19:42:59.589047 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:59.589055 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:59.589285 (+ 230us) server_negotiation.cc:407] Connection header received
0806 19:42:59.589544 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:59.589552 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:59.589682 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:59.589790 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:59.591319 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.592088 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.593479 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.593896 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.594067 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.599996 (+ 5929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47722: BlockingRecv error: recv got EOF from 127.0.0.1:47722 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 06 19:42:59 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045379, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
I20260806 19:42:59.619266 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.603978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47734 (local address 127.21.202.126:46343)
0806 19:42:59.604220 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:59.604228 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:59.604518 (+ 290us) server_negotiation.cc:407] Connection header received
0806 19:42:59.604736 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:59.604746 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:59.604861 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:59.604979 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:42:59.606172 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.607083 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.608558 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.609169 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.609312 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.613575 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:42:59.613582 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:42:59.613586 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:42:59.613649 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:42:59.616462 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:59.617235 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:59.617241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:59.617245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:59.617381 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:42:59.617789 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:42:59.617795 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:42:59.617798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:42:59.618098 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:42:59.618379 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:59.618698 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 19:42:59.618930 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 19:42:59.623797 30051 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:42:59.632030 30051 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:42:59.660323 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.648706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55476 (local address 127.21.202.66:42377)
0806 19:42:59.650364 (+ 1658us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:59.650371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:59.650992 (+ 621us) server_negotiation.cc:407] Connection header received
0806 19:42:59.651316 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:59.651326 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:59.651577 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:59.651726 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:59.652696 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.654332 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.657088 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.658539 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.658659 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.658923 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:59.659747 (+ 824us) server_negotiation.cc:299] Negotiation successful
0806 19:42:59.659960 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1507,"thread_start_us":1352,"threads_started":1}
I20260806 19:42:59.660953 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.646841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:44865 (local address 127.0.0.1:57080)
0806 19:42:59.648756 (+ 1915us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:59.648852 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:59.649140 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:59.650882 (+ 1742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:59.650889 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:59.650941 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:59.651459 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.651473 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.654164 (+ 2691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.654170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.656859 (+ 2689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.656877 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.659919 (+ 3042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.659926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.660121 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.660161 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 19:42:59.660231 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 19:42:59.660462 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"spinlock_wait_cycles":82176,"thread_start_us":631,"threads_started":1}
I20260806 19:42:59.661630 30562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.648279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:42377 (local address 127.0.0.1:55476)
0806 19:42:59.650890 (+ 2611us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:59.650908 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:59.651118 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:59.651992 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:59.651999 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:59.652025 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:59.652490 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.652503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.654557 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.654562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.656906 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.656919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.659123 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.659132 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.659413 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.659488 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 19:42:59.660622 (+ 1134us) client_negotiation.cc:240] Negotiation successful
0806 19:42:59.660764 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2459,"thread_start_us":2282,"threads_started":1}
I20260806 19:42:59.661979 30561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.647296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57080 (local address 127.21.202.67:44865)
0806 19:42:59.650161 (+ 2865us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:59.650171 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:59.650207 (+ 36us) server_negotiation.cc:407] Connection header received
0806 19:42:59.650316 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:59.650325 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:59.650616 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:59.651691 (+ 1075us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:59.652040 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.653998 (+ 1958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.657118 (+ 3120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.658539 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.661076 (+ 2537us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.661350 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:59.661435 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 19:42:59.661625 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2659,"thread_start_us":2415,"threads_started":1}
I20260806 19:42:59.666572 30468 tablet_service.cc:1467] Processing CreateTablet for tablet 1d8a4d045f044ee8ac99b3b8ccf53f12 (DEFAULT_TABLE table=test-table [id=41eaf0700a254cd3b540c4fda26bd937]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:59.666572 30328 tablet_service.cc:1467] Processing CreateTablet for tablet 1d8a4d045f044ee8ac99b3b8ccf53f12 (DEFAULT_TABLE table=test-table [id=41eaf0700a254cd3b540c4fda26bd937]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:59.668151 30468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d8a4d045f044ee8ac99b3b8ccf53f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:59.668154 30328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d8a4d045f044ee8ac99b3b8ccf53f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:59.668725 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.651843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37773 (local address 127.0.0.1:40380)
0806 19:42:59.661186 (+ 9343us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:59.661207 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:59.661315 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:59.663000 (+ 1685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:59.663008 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:59.663038 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:59.663191 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.663201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.664762 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.664769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.666353 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.666367 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.667905 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.667910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.668095 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.668136 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 19:42:59.668266 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 19:42:59.668420 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9244,"thread_start_us":10549,"threads_started":1}
I20260806 19:42:59.668790 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.651210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40380 (local address 127.21.202.65:37773)
0806 19:42:59.662248 (+ 11038us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:59.662259 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:59.662295 (+ 36us) server_negotiation.cc:407] Connection header received
0806 19:42:59.662415 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:59.662427 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:59.662762 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:59.662995 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:59.663352 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.664607 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.666537 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.667711 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.667874 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.668083 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:59.668322 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 19:42:59.668485 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10859,"thread_start_us":10722,"threads_started":1}
I20260806 19:42:59.672325 30189 tablet_service.cc:1467] Processing CreateTablet for tablet 1d8a4d045f044ee8ac99b3b8ccf53f12 (DEFAULT_TABLE table=test-table [id=41eaf0700a254cd3b540c4fda26bd937]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:42:59.674211 30189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d8a4d045f044ee8ac99b3b8ccf53f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:42:59.685451 30568 tablet_bootstrap.cc:492] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149: Bootstrap starting.
I20260806 19:42:59.688020 30566 tablet_bootstrap.cc:492] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e: Bootstrap starting.
I20260806 19:42:59.689597 30567 tablet_bootstrap.cc:492] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3: Bootstrap starting.
I20260806 19:42:59.691025 30568 tablet_bootstrap.cc:654] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:59.691923 30566 tablet_bootstrap.cc:654] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:59.692569 30568 log.cc:824] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:59.693219 30566 log.cc:824] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:59.693395 30567 tablet_bootstrap.cc:654] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3: Neither blocks nor log segments found. Creating new log.
I20260806 19:42:59.694819 30567 log.cc:824] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3: Log is configured to *not* fsync() on all Append() calls
I20260806 19:42:59.696437 30568 tablet_bootstrap.cc:492] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149: No bootstrap required, opened a new log
I20260806 19:42:59.696764 30566 tablet_bootstrap.cc:492] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e: No bootstrap required, opened a new log
I20260806 19:42:59.696759 30568 ts_tablet_manager.cc:1397] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.004s
I20260806 19:42:59.697137 30566 ts_tablet_manager.cc:1397] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260806 19:42:59.699481 30567 tablet_bootstrap.cc:492] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3: No bootstrap required, opened a new log
I20260806 19:42:59.699786 30567 ts_tablet_manager.cc:1397] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 19:42:59.707129 30568 raft_consensus.cc:348] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } }
I20260806 19:42:59.707683 30568 raft_consensus.cc:374] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:59.707806 30568 raft_consensus.cc:729] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc7f144fed494853a610eac95b529149, State: Initialized, Role: FOLLOWER
I20260806 19:42:59.709029 30568 consensus_queue.cc:260] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } }
I20260806 19:42:59.709427 30567 raft_consensus.cc:348] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } }
I20260806 19:42:59.709950 30567 raft_consensus.cc:374] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:59.710064 30567 raft_consensus.cc:729] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec750bd3403b47cba074d1b138e717a3, State: Initialized, Role: FOLLOWER
I20260806 19:42:59.710189 30566 raft_consensus.cc:348] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } }
I20260806 19:42:59.710680 30566 raft_consensus.cc:374] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:42:59.710806 30566 raft_consensus.cc:729] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 495b6134be8e4b7db72d06fe0900920e, State: Initialized, Role: FOLLOWER
I20260806 19:42:59.711246 30568 ts_tablet_manager.cc:1428] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149: Time spent starting tablet: real 0.014s user 0.010s sys 0.002s
I20260806 19:42:59.711539 30566 consensus_queue.cc:260] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } }
I20260806 19:42:59.712281 30567 consensus_queue.cc:260] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } }
I20260806 19:42:59.713670 30566 ts_tablet_manager.cc:1428] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
I20260806 19:42:59.714149 30567 ts_tablet_manager.cc:1428] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260806 19:42:59.714169 30398 heartbeater.cc:503] Master 127.21.202.126:46343 was elected leader, sending a full tablet report...
I20260806 19:42:59.714538 30541 heartbeater.cc:503] Master 127.21.202.126:46343 was elected leader, sending a full tablet report...
W20260806 19:42:59.719129 30399 tablet.cc:2369] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:42:59.779907 30542 tablet.cc:2369] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:42:59.910562 30573 raft_consensus.cc:482] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:42:59.910943 30573 raft_consensus.cc:504] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } }
I20260806 19:42:59.912794 30573 leader_election.cc:288] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec750bd3403b47cba074d1b138e717a3 (127.21.202.67:44865), cc7f144fed494853a610eac95b529149 (127.21.202.65:37773)
W20260806 19:42:59.917574 30260 tablet.cc:2369] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:42:59.920796 30561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.913532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:46415 (local address 127.21.202.67:44865)
0806 19:42:59.913765 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:59.913771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:59.914126 (+ 355us) server_negotiation.cc:407] Connection header received
0806 19:42:59.914350 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:59.914355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:59.914448 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:59.914572 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:59.915390 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.916403 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.918566 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.919816 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.919947 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.920000 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:59.920480 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 19:42:59.920592 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 19:42:59.920969 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.913248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:44865 (local address 127.21.202.66:46415)
0806 19:42:59.913996 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:59.914017 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:59.914164 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:59.914684 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:59.914690 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:59.914732 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:59.915203 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.915219 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.916561 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.916568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.918408 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.918421 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.920065 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.920072 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.920257 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.920294 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:42:59.920412 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 19:42:59.920575 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":379,"threads_started":1}
I20260806 19:42:59.921983 30576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.914098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37773 (local address 127.21.202.66:48981)
0806 19:42:59.914813 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0806 19:42:59.914827 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 19:42:59.914942 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:42:59.915390 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:42:59.915395 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:42:59.915422 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:42:59.915849 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.915861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.917264 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.917269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.919452 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:42:59.919467 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.921303 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.921310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:42:59.921506 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.921537 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 19:42:59.921646 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 19:42:59.921753 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":404,"threads_started":1}
I20260806 19:42:59.922206 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.914338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:48981 (local address 127.21.202.65:37773)
0806 19:42:59.914577 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:59.914584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:59.914937 (+ 353us) server_negotiation.cc:407] Connection header received
0806 19:42:59.915061 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:59.915069 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:59.915184 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:59.915300 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:42:59.916068 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.917072 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.919641 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.920886 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.921072 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.921148 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:59.921723 (+ 575us) server_negotiation.cc:299] Negotiation successful
0806 19:42:59.921936 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:42:59.922240 30489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d8a4d045f044ee8ac99b3b8ccf53f12" candidate_uuid: "495b6134be8e4b7db72d06fe0900920e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec750bd3403b47cba074d1b138e717a3" is_pre_election: true
I20260806 19:42:59.922901 30489 raft_consensus.cc:2454] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 495b6134be8e4b7db72d06fe0900920e in term 0.
I20260806 19:42:59.923154 30209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d8a4d045f044ee8ac99b3b8ccf53f12" candidate_uuid: "495b6134be8e4b7db72d06fe0900920e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc7f144fed494853a610eac95b529149" is_pre_election: true
I20260806 19:42:59.923641 30280 leader_election.cc:302] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 495b6134be8e4b7db72d06fe0900920e, ec750bd3403b47cba074d1b138e717a3; no voters:
I20260806 19:42:59.923693 30209 raft_consensus.cc:2454] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 495b6134be8e4b7db72d06fe0900920e in term 0.
I20260806 19:42:59.924100 30573 raft_consensus.cc:2781] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:42:59.924254 30573 raft_consensus.cc:482] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:42:59.924384 30573 raft_consensus.cc:3037] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:59.926822 30573 raft_consensus.cc:504] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } }
I20260806 19:42:59.927562 30573 leader_election.cc:288] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [CANDIDATE]: Term 1 election: Requested vote from peers ec750bd3403b47cba074d1b138e717a3 (127.21.202.67:44865), cc7f144fed494853a610eac95b529149 (127.21.202.65:37773)
I20260806 19:42:59.928145 30489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d8a4d045f044ee8ac99b3b8ccf53f12" candidate_uuid: "495b6134be8e4b7db72d06fe0900920e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec750bd3403b47cba074d1b138e717a3"
I20260806 19:42:59.928198 30209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d8a4d045f044ee8ac99b3b8ccf53f12" candidate_uuid: "495b6134be8e4b7db72d06fe0900920e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc7f144fed494853a610eac95b529149"
I20260806 19:42:59.928378 30489 raft_consensus.cc:3037] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:59.928432 30209 raft_consensus.cc:3037] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:42:59.930925 30489 raft_consensus.cc:2454] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 495b6134be8e4b7db72d06fe0900920e in term 1.
I20260806 19:42:59.931205 30209 raft_consensus.cc:2454] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 495b6134be8e4b7db72d06fe0900920e in term 1.
I20260806 19:42:59.931416 30280 leader_election.cc:302] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 495b6134be8e4b7db72d06fe0900920e, ec750bd3403b47cba074d1b138e717a3; no voters:
I20260806 19:42:59.931735 30573 raft_consensus.cc:2781] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:42:59.932276 30573 raft_consensus.cc:686] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [term 1 LEADER]: Becoming Leader. State: Replica: 495b6134be8e4b7db72d06fe0900920e, State: Running, Role: LEADER
I20260806 19:42:59.932935 30573 consensus_queue.cc:237] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } }
I20260806 19:42:59.937942 30049 catalog_manager.cc:5495] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e reported cstate change: term changed from 0 to 1, leader changed from <none> to 495b6134be8e4b7db72d06fe0900920e (127.21.202.66). New cstate: current_term: 1 leader_uuid: "495b6134be8e4b7db72d06fe0900920e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } health_report { overall_health: UNKNOWN } } }
I20260806 19:42:59.961334 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:42:59.955450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55486 (local address 127.21.202.66:42377)
0806 19:42:59.955659 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 19:42:59.955667 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:42:59.955709 (+ 42us) server_negotiation.cc:407] Connection header received
0806 19:42:59.956070 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:42:59.956075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:42:59.956155 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:42:59.956285 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:42:59.956681 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.957460 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.958961 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:42:59.959394 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:42:59.959522 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:42:59.960023 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:42:59.960594 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:42:59.960710 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 19:42:59.960941 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 19:42:59.961017 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 19:42:59.971060 30209 raft_consensus.cc:1261] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Refusing update from remote peer 495b6134be8e4b7db72d06fe0900920e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:59.972025 30489 raft_consensus.cc:1261] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Refusing update from remote peer 495b6134be8e4b7db72d06fe0900920e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:42:59.972254 30577 consensus_queue.cc:1035] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:59.972997 30573 consensus_queue.cc:1035] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:42:59.985787 30580 mvcc.cc:205] Tried to move back new op lower bound from 7315641876349345792 to 7315641876209856512. Current Snapshot: MvccSnapshot[applied={T|T < 7315641876349345792}]
I20260806 19:42:59.989604 30581 mvcc.cc:205] Tried to move back new op lower bound from 7315641876349345792 to 7315641876209856512. Current Snapshot: MvccSnapshot[applied={T|T < 7315641876349345792}]
I20260806 19:43:00.002261 30049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:43:00.002625 30049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:43:00.007509 30049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61: Sending DeleteTablet for 3 replicas of tablet 1d8a4d045f044ee8ac99b3b8ccf53f12
I20260806 19:43:00.009020 30468 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d8a4d045f044ee8ac99b3b8ccf53f12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:59 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:57080
I20260806 19:43:00.009302 30189 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d8a4d045f044ee8ac99b3b8ccf53f12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:59 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:40380
I20260806 19:43:00.012255 30328 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d8a4d045f044ee8ac99b3b8ccf53f12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:42:59 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:55476
I20260806 19:43:00.016410 30049 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:43:00.017594 30049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:43:00.018924 30592 tablet_replica.cc:330] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e: stopping tablet replica
I20260806 19:43:00.019531 30592 raft_consensus.cc:2229] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:43:00.020416 30592 raft_consensus.cc:2258] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:00.027375 30590 tablet_replica.cc:330] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3: stopping tablet replica
I20260806 19:43:00.027961 30590 raft_consensus.cc:2229] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:43:00.028465 30590 raft_consensus.cc:2258] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:00.029544 30188 tablet_service.cc:1467] Processing CreateTablet for tablet 2bee81cfbf57409982db0434f0beb764 (DEFAULT_TABLE table=test-table [id=a81418f109fd42d48871f55a2de7ed99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:43:00.030234 30188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bee81cfbf57409982db0434f0beb764. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:00.030601 30590 ts_tablet_manager.cc:1905] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:00.034541 30468 tablet_service.cc:1467] Processing CreateTablet for tablet 2bee81cfbf57409982db0434f0beb764 (DEFAULT_TABLE table=test-table [id=a81418f109fd42d48871f55a2de7ed99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:43:00.035477 30468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bee81cfbf57409982db0434f0beb764. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:00.033349 30328 tablet_service.cc:1467] Processing CreateTablet for tablet 2bee81cfbf57409982db0434f0beb764 (DEFAULT_TABLE table=test-table [id=a81418f109fd42d48871f55a2de7ed99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:43:00.037705 30590 ts_tablet_manager.cc:1918] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:43:00.037950 30590 log.cc:1196] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/1d8a4d045f044ee8ac99b3b8ccf53f12
I20260806 19:43:00.038676 30590 ts_tablet_manager.cc:1939] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P ec750bd3403b47cba074d1b138e717a3: Deleting consensus metadata
I20260806 19:43:00.039208 30328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bee81cfbf57409982db0434f0beb764. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:00.040485 30592 ts_tablet_manager.cc:1905] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:00.041007 30036 catalog_manager.cc:4841] TS ec750bd3403b47cba074d1b138e717a3 (127.21.202.67:44865): tablet 1d8a4d045f044ee8ac99b3b8ccf53f12 (table test-table [id=41eaf0700a254cd3b540c4fda26bd937]) successfully deleted
I20260806 19:43:00.042541 30568 tablet_bootstrap.cc:492] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149: Bootstrap starting.
I20260806 19:43:00.042866 30591 tablet_replica.cc:330] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149: stopping tablet replica
I20260806 19:43:00.043377 30591 raft_consensus.cc:2229] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:43:00.043990 30591 raft_consensus.cc:2258] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:00.044878 30568 tablet_bootstrap.cc:654] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:00.046571 30591 ts_tablet_manager.cc:1905] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:00.047683 30566 tablet_bootstrap.cc:492] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e: Bootstrap starting.
I20260806 19:43:00.049901 30566 tablet_bootstrap.cc:654] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:00.049957 30592 ts_tablet_manager.cc:1918] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:43:00.050266 30592 log.cc:1196] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/1d8a4d045f044ee8ac99b3b8ccf53f12
I20260806 19:43:00.051146 30592 ts_tablet_manager.cc:1939] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P 495b6134be8e4b7db72d06fe0900920e: Deleting consensus metadata
I20260806 19:43:00.053146 30036 catalog_manager.cc:4841] TS 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377): tablet 1d8a4d045f044ee8ac99b3b8ccf53f12 (table test-table [id=41eaf0700a254cd3b540c4fda26bd937]) successfully deleted
I20260806 19:43:00.054363 30591 ts_tablet_manager.cc:1918] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:43:00.054535 30591 log.cc:1196] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/1d8a4d045f044ee8ac99b3b8ccf53f12
I20260806 19:43:00.055187 30591 ts_tablet_manager.cc:1939] T 1d8a4d045f044ee8ac99b3b8ccf53f12 P cc7f144fed494853a610eac95b529149: Deleting consensus metadata
I20260806 19:43:00.057124 30567 tablet_bootstrap.cc:492] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3: Bootstrap starting.
I20260806 19:43:00.057433 30036 catalog_manager.cc:4841] TS cc7f144fed494853a610eac95b529149 (127.21.202.65:37773): tablet 1d8a4d045f044ee8ac99b3b8ccf53f12 (table test-table [id=41eaf0700a254cd3b540c4fda26bd937]) successfully deleted
I20260806 19:43:00.058992 30567 tablet_bootstrap.cc:654] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:00.061216 30567 tablet_bootstrap.cc:492] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3: No bootstrap required, opened a new log
I20260806 19:43:00.061384 30567 ts_tablet_manager.cc:1397] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260806 19:43:00.061735 30568 tablet_bootstrap.cc:492] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149: No bootstrap required, opened a new log
I20260806 19:43:00.061980 30568 ts_tablet_manager.cc:1397] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.001s
I20260806 19:43:00.062106 30566 tablet_bootstrap.cc:492] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e: No bootstrap required, opened a new log
I20260806 19:43:00.062332 30566 ts_tablet_manager.cc:1397] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.007s
I20260806 19:43:00.062243 30567 raft_consensus.cc:348] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.062510 30567 raft_consensus.cc:374] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:00.062614 30567 raft_consensus.cc:729] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec750bd3403b47cba074d1b138e717a3, State: Initialized, Role: FOLLOWER
I20260806 19:43:00.062943 30567 consensus_queue.cc:260] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.063283 30568 raft_consensus.cc:348] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.063568 30568 raft_consensus.cc:374] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:00.063431 30566 raft_consensus.cc:348] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.063735 30566 raft_consensus.cc:374] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:00.063756 30568 raft_consensus.cc:729] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc7f144fed494853a610eac95b529149, State: Initialized, Role: FOLLOWER
I20260806 19:43:00.064316 30567 ts_tablet_manager.cc:1428] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 19:43:00.064185 30568 consensus_queue.cc:260] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.064890 30566 raft_consensus.cc:729] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 495b6134be8e4b7db72d06fe0900920e, State: Initialized, Role: FOLLOWER
I20260806 19:43:00.065052 30568 ts_tablet_manager.cc:1428] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 19:43:00.065270 30566 consensus_queue.cc:260] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.066067 30566 ts_tablet_manager.cc:1428] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260806 19:43:00.141175 30572 raft_consensus.cc:482] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:43:00.141469 30572 raft_consensus.cc:504] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.143244 30572 leader_election.cc:288] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377), ec750bd3403b47cba074d1b138e717a3 (127.21.202.67:44865)
I20260806 19:43:00.151692 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.143666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:47071 (local address 127.21.202.66:42377)
0806 19:43:00.143927 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:00.143936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:00.144575 (+ 639us) server_negotiation.cc:407] Connection header received
0806 19:43:00.144985 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:00.145001 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:00.145148 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:00.145246 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:43:00.146204 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.147398 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.149410 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.150400 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.150627 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.150715 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:00.151172 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 19:43:00.151336 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 19:43:00.151767 30597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.143698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:42377 (local address 127.21.202.65:47071)
0806 19:43:00.144466 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:00.144490 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:00.144702 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:00.145487 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:00.145490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:00.145515 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:43:00.145984 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.145999 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.147609 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.147621 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.149235 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.149249 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.150673 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.150681 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.150990 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.151020 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 19:43:00.151141 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 19:43:00.151293 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"mutex_wait_us":76,"thread_start_us":394,"threads_started":1}
I20260806 19:43:00.153227 30352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bee81cfbf57409982db0434f0beb764" candidate_uuid: "cc7f144fed494853a610eac95b529149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495b6134be8e4b7db72d06fe0900920e" is_pre_election: true
I20260806 19:43:00.153745 30561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.144827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:32981 (local address 127.21.202.67:44865)
0806 19:43:00.145032 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:00.145039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:00.145790 (+ 751us) server_negotiation.cc:407] Connection header received
0806 19:43:00.146077 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:00.146084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:00.146164 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:00.146245 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:43:00.147169 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.148339 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.151231 (+ 2892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.152532 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.152718 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.152845 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:00.153270 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 19:43:00.153423 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 19:43:00.153808 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.144619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:44865 (local address 127.21.202.65:32981)
0806 19:43:00.145603 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:00.145629 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:00.145874 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:00.146495 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:00.146501 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:00.146543 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:43:00.146946 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.146958 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.148546 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.148551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.151014 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.151033 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.152785 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.152793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.153059 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.153099 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 19:43:00.153214 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 19:43:00.153369 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":35,"thread_start_us":594,"threads_started":1}
I20260806 19:43:00.153798 30352 raft_consensus.cc:2454] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc7f144fed494853a610eac95b529149 in term 0.
I20260806 19:43:00.154488 30489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bee81cfbf57409982db0434f0beb764" candidate_uuid: "cc7f144fed494853a610eac95b529149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec750bd3403b47cba074d1b138e717a3" is_pre_election: true
I20260806 19:43:00.154794 30489 raft_consensus.cc:2454] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc7f144fed494853a610eac95b529149 in term 0.
I20260806 19:43:00.154742 30139 leader_election.cc:302] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 495b6134be8e4b7db72d06fe0900920e, cc7f144fed494853a610eac95b529149; no voters:
I20260806 19:43:00.155258 30572 raft_consensus.cc:2781] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:43:00.155442 30572 raft_consensus.cc:482] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:43:00.155561 30572 raft_consensus.cc:3037] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:00.158059 30572 raft_consensus.cc:504] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.158803 30572 leader_election.cc:288] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [CANDIDATE]: Term 1 election: Requested vote from peers 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377), ec750bd3403b47cba074d1b138e717a3 (127.21.202.67:44865)
I20260806 19:43:00.159428 30352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bee81cfbf57409982db0434f0beb764" candidate_uuid: "cc7f144fed494853a610eac95b529149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495b6134be8e4b7db72d06fe0900920e"
I20260806 19:43:00.159435 30489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bee81cfbf57409982db0434f0beb764" candidate_uuid: "cc7f144fed494853a610eac95b529149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec750bd3403b47cba074d1b138e717a3"
I20260806 19:43:00.159670 30352 raft_consensus.cc:3037] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:00.159672 30489 raft_consensus.cc:3037] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:00.162563 30352 raft_consensus.cc:2454] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc7f144fed494853a610eac95b529149 in term 1.
I20260806 19:43:00.162923 30489 raft_consensus.cc:2454] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc7f144fed494853a610eac95b529149 in term 1.
I20260806 19:43:00.163075 30139 leader_election.cc:302] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 495b6134be8e4b7db72d06fe0900920e, cc7f144fed494853a610eac95b529149; no voters:
I20260806 19:43:00.163457 30572 raft_consensus.cc:2781] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:43:00.164048 30572 raft_consensus.cc:686] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [term 1 LEADER]: Becoming Leader. State: Replica: cc7f144fed494853a610eac95b529149, State: Running, Role: LEADER
I20260806 19:43:00.164719 30572 consensus_queue.cc:237] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.168895 30049 catalog_manager.cc:5495] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc7f144fed494853a610eac95b529149 (127.21.202.65). New cstate: current_term: 1 leader_uuid: "cc7f144fed494853a610eac95b529149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } health_report { overall_health: UNKNOWN } } }
Aug 06 19:43:00 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045377, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.126@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
I20260806 19:43:00.259243 30607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.235973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46343 (local address 127.0.0.1:47750)
0806 19:43:00.236961 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:00.236979 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:00.237176 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:00.237717 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:00.237721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:00.238554 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:00.238919 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.238931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.240792 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.240799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.242281 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.242297 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.243320 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.243325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.243471 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.249127 (+ 5656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:00.249151 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:00.251668 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:43:00.256182 (+ 4514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:00.256195 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:00.256202 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:00.256653 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:00.257219 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:00.257225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:00.257229 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:00.257456 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:00.258209 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:43:00.258217 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:43:00.258401 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 19:43:00.258563 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 19:43:00.258799 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"thread_start_us":677,"threads_started":1}
I20260806 19:43:00.259682 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.235943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47750 (local address 127.21.202.126:46343)
0806 19:43:00.236781 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:00.236793 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:00.237084 (+ 291us) server_negotiation.cc:407] Connection header received
0806 19:43:00.237376 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:00.237383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:00.237500 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:00.237624 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:00.239103 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.240516 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.242453 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.243135 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.243268 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.251979 (+ 8711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:00.251988 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:00.251993 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:00.252038 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:00.256012 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:00.256840 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:00.256847 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:00.256852 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:00.257031 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:00.257620 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:00.257626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:00.257629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:00.258031 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:00.258219 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:00.258725 (+ 506us) server_negotiation.cc:299] Negotiation successful
0806 19:43:00.258969 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":411,"threads_started":1}
I20260806 19:43:00.265291 30049 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:47750:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 19:43:00.266541 30049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:43:00.274438 30188 tablet_service.cc:1467] Processing CreateTablet for tablet bdbd6c5b037941168197069cc7160fb8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1465f21d6c174ffbad0613cd147d1c4c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:43:00.274629 30328 tablet_service.cc:1467] Processing CreateTablet for tablet bdbd6c5b037941168197069cc7160fb8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1465f21d6c174ffbad0613cd147d1c4c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:43:00.275107 30188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdbd6c5b037941168197069cc7160fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:00.275027 30468 tablet_service.cc:1467] Processing CreateTablet for tablet bdbd6c5b037941168197069cc7160fb8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1465f21d6c174ffbad0613cd147d1c4c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:43:00.275290 30328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdbd6c5b037941168197069cc7160fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:00.275586 30468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdbd6c5b037941168197069cc7160fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:00.284538 30566 tablet_bootstrap.cc:492] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e: Bootstrap starting.
I20260806 19:43:00.285184 30567 tablet_bootstrap.cc:492] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3: Bootstrap starting.
I20260806 19:43:00.285737 30568 tablet_bootstrap.cc:492] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149: Bootstrap starting.
I20260806 19:43:00.287534 30567 tablet_bootstrap.cc:654] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:00.288097 30568 tablet_bootstrap.cc:654] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:00.291667 30566 tablet_bootstrap.cc:654] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:00.296250 30568 tablet_bootstrap.cc:492] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149: No bootstrap required, opened a new log
I20260806 19:43:00.296509 30568 ts_tablet_manager.cc:1397] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260806 19:43:00.297576 30568 raft_consensus.cc:348] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.297881 30568 raft_consensus.cc:374] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:00.297978 30568 raft_consensus.cc:729] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc7f144fed494853a610eac95b529149, State: Initialized, Role: FOLLOWER
I20260806 19:43:00.298257 30568 consensus_queue.cc:260] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.298946 30568 ts_tablet_manager.cc:1428] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 19:43:00.298947 30567 tablet_bootstrap.cc:492] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3: No bootstrap required, opened a new log
I20260806 19:43:00.299365 30566 tablet_bootstrap.cc:492] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e: No bootstrap required, opened a new log
I20260806 19:43:00.299595 30566 ts_tablet_manager.cc:1397] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.002s
I20260806 19:43:00.298749 30572 tablet_replica.cc:439] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } }
I20260806 19:43:00.299868 30572 tablet_replica.cc:442] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:43:00.300603 30566 raft_consensus.cc:348] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.300904 30566 raft_consensus.cc:374] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:00.300987 30566 raft_consensus.cc:729] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 495b6134be8e4b7db72d06fe0900920e, State: Initialized, Role: FOLLOWER
I20260806 19:43:00.301244 30566 consensus_queue.cc:260] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.301631 30567 ts_tablet_manager.cc:1397] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.006s
I20260806 19:43:00.302423 30567 raft_consensus.cc:348] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.302673 30567 raft_consensus.cc:374] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:00.302750 30567 raft_consensus.cc:729] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec750bd3403b47cba074d1b138e717a3, State: Initialized, Role: FOLLOWER
I20260806 19:43:00.302985 30567 consensus_queue.cc:260] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.303452 30586 tablet_replica.cc:439] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } }
I20260806 19:43:00.303812 30586 tablet_replica.cc:442] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:43:00.304314 30566 ts_tablet_manager.cc:1428] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 19:43:00.304713 30574 tablet_replica.cc:439] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } }
I20260806 19:43:00.305033 30574 tablet_replica.cc:442] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:43:00.305990 30567 ts_tablet_manager.cc:1428] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260806 19:43:00.405287 30572 raft_consensus.cc:482] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:43:00.405544 30572 raft_consensus.cc:504] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.406311 30572 leader_election.cc:288] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377), ec750bd3403b47cba074d1b138e717a3 (127.21.202.67:44865)
I20260806 19:43:00.407092 30352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdbd6c5b037941168197069cc7160fb8" candidate_uuid: "cc7f144fed494853a610eac95b529149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495b6134be8e4b7db72d06fe0900920e" is_pre_election: true
I20260806 19:43:00.407104 30489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdbd6c5b037941168197069cc7160fb8" candidate_uuid: "cc7f144fed494853a610eac95b529149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec750bd3403b47cba074d1b138e717a3" is_pre_election: true
I20260806 19:43:00.407407 30352 raft_consensus.cc:2454] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc7f144fed494853a610eac95b529149 in term 0.
I20260806 19:43:00.407414 30489 raft_consensus.cc:2454] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc7f144fed494853a610eac95b529149 in term 0.
I20260806 19:43:00.407991 30139 leader_election.cc:302] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 495b6134be8e4b7db72d06fe0900920e, cc7f144fed494853a610eac95b529149; no voters:
I20260806 19:43:00.408339 30572 raft_consensus.cc:2781] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:43:00.408481 30572 raft_consensus.cc:482] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:43:00.408697 30572 raft_consensus.cc:3037] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:00.412385 30572 raft_consensus.cc:504] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.413268 30572 leader_election.cc:288] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [CANDIDATE]: Term 1 election: Requested vote from peers 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377), ec750bd3403b47cba074d1b138e717a3 (127.21.202.67:44865)
I20260806 19:43:00.413908 30352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdbd6c5b037941168197069cc7160fb8" candidate_uuid: "cc7f144fed494853a610eac95b529149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495b6134be8e4b7db72d06fe0900920e"
I20260806 19:43:00.414162 30352 raft_consensus.cc:3037] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:00.414323 30489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdbd6c5b037941168197069cc7160fb8" candidate_uuid: "cc7f144fed494853a610eac95b529149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec750bd3403b47cba074d1b138e717a3"
I20260806 19:43:00.414570 30489 raft_consensus.cc:3037] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:00.417359 30352 raft_consensus.cc:2454] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc7f144fed494853a610eac95b529149 in term 1.
I20260806 19:43:00.417582 30489 raft_consensus.cc:2454] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc7f144fed494853a610eac95b529149 in term 1.
I20260806 19:43:00.417860 30139 leader_election.cc:302] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 495b6134be8e4b7db72d06fe0900920e, cc7f144fed494853a610eac95b529149; no voters:
I20260806 19:43:00.418429 30572 raft_consensus.cc:2781] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:43:00.419013 30572 raft_consensus.cc:686] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [term 1 LEADER]: Becoming Leader. State: Replica: cc7f144fed494853a610eac95b529149, State: Running, Role: LEADER
I20260806 19:43:00.419281 30572 consensus_queue.cc:237] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.421532 30599 tablet_replica.cc:439] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149: TxnStatusTablet state changed. Reason: New leader cc7f144fed494853a610eac95b529149. Latest consensus state: current_term: 1 leader_uuid: "cc7f144fed494853a610eac95b529149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } }
I20260806 19:43:00.421926 30599 tablet_replica.cc:442] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:43:00.422991 30612 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 19:43:00.423235 30612 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 19:43:00.424497 30051 catalog_manager.cc:5495] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc7f144fed494853a610eac95b529149 (127.21.202.65). New cstate: current_term: 1 leader_uuid: "cc7f144fed494853a610eac95b529149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } health_report { overall_health: UNKNOWN } } }
I20260806 19:43:00.469564 30607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.460221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37773 (local address 127.0.0.1:40392)
0806 19:43:00.460677 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:00.460717 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:00.460913 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:00.462118 (+ 1205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:00.462123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:00.462139 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:43:00.462362 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.462374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.464080 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.464085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.465776 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.465794 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.467076 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.467086 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.467328 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.467429 (+ 101us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:43:00.468862 (+ 1433us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:43:00.468892 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 19:43:00.469009 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 19:43:00.469206 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258}
I20260806 19:43:00.469935 30615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.460196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.21.202.65:37773)
0806 19:43:00.461578 (+ 1382us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:00.461586 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:00.461616 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:43:00.461747 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:00.461753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:00.461873 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:00.462072 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:43:00.462563 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.463898 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.465983 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.466766 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.466928 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.467752 (+ 824us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:43:00.468662 (+ 910us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:43:00.468875 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:00.469078 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 19:43:00.469184 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1224,"thread_start_us":1025,"threads_started":1}
I20260806 19:43:00.475142 30489 raft_consensus.cc:1261] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Refusing update from remote peer cc7f144fed494853a610eac95b529149: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:43:00.475548 30352 raft_consensus.cc:1261] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Refusing update from remote peer cc7f144fed494853a610eac95b529149: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:43:00.476042 30599 consensus_queue.cc:1035] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:43:00.476574 30572 consensus_queue.cc:1035] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:43:00.489837 30586 tablet_replica.cc:439] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e: TxnStatusTablet state changed. Reason: New leader cc7f144fed494853a610eac95b529149. Latest consensus state: current_term: 1 leader_uuid: "cc7f144fed494853a610eac95b529149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } }
I20260806 19:43:00.490197 30586 tablet_replica.cc:442] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:43:00.490572 30574 tablet_replica.cc:439] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3: TxnStatusTablet state changed. Reason: New leader cc7f144fed494853a610eac95b529149. Latest consensus state: current_term: 1 leader_uuid: "cc7f144fed494853a610eac95b529149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } }
I20260806 19:43:00.490857 30574 tablet_replica.cc:442] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:43:00.494608 30599 tablet_replica.cc:439] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc7f144fed494853a610eac95b529149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } }
I20260806 19:43:00.494925 30599 tablet_replica.cc:442] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:43:00.514436 30574 tablet_replica.cc:439] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc7f144fed494853a610eac95b529149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } }
I20260806 19:43:00.514837 30574 tablet_replica.cc:442] T bdbd6c5b037941168197069cc7160fb8 P ec750bd3403b47cba074d1b138e717a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:43:00.516445 30599 tablet_replica.cc:439] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc7f144fed494853a610eac95b529149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } }
I20260806 19:43:00.516723 30599 tablet_replica.cc:442] T bdbd6c5b037941168197069cc7160fb8 P cc7f144fed494853a610eac95b529149: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:43:00.522863 30586 tablet_replica.cc:439] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc7f144fed494853a610eac95b529149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } } }
I20260806 19:43:00.523255 30586 tablet_replica.cc:442] T bdbd6c5b037941168197069cc7160fb8 P 495b6134be8e4b7db72d06fe0900920e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:43:00.537017 30615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.511986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40402 (local address 127.21.202.65:37773)
0806 19:43:00.512177 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:00.512183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:00.529236 (+ 17053us) server_negotiation.cc:407] Connection header received
0806 19:43:00.529580 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:00.529586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:00.529689 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:00.529782 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:43:00.530681 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.531977 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.534355 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.535198 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.535323 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.536038 (+ 715us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:43:00.536193 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:43:00.536266 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:00.536511 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 19:43:00.536613 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 19:43:00.575776 30618 consensus_queue.cc:1035] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:43:00.576457 30615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.569403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:40255 (local address 127.21.202.65:37773)
0806 19:43:00.569654 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:00.569662 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:00.570231 (+ 569us) server_negotiation.cc:407] Connection header received
0806 19:43:00.570403 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:00.570408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:00.570537 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:00.570637 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:43:00.571273 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.572466 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.574054 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.574735 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.574876 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.575593 (+ 717us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:43:00.575753 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:43:00.575851 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:00.576031 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 19:43:00.576114 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 19:43:00.576628 30625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.569440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37773 (local address 127.21.202.65:40255)
0806 19:43:00.570106 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:00.570127 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:00.570265 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:00.570682 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:00.570685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:00.570696 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:43:00.571074 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.571083 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.572619 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.572624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.573896 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.573909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.574940 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.574946 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.575152 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.575247 (+ 95us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:43:00.575870 (+ 623us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:43:00.575882 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 19:43:00.575976 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 19:43:00.576130 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":389,"threads_started":1}
I20260806 19:43:00.589215 30489 raft_consensus.cc:1261] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Refusing update from remote peer cc7f144fed494853a610eac95b529149: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:43:00.590161 30618 consensus_queue.cc:1035] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:43:00.672875 30049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:43:00.673228 30049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:43:00.676497 30049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61: Sending DeleteTablet for 3 replicas of tablet 2bee81cfbf57409982db0434f0beb764
I20260806 19:43:00.677626 30468 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bee81cfbf57409982db0434f0beb764 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:43:00 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:57080
I20260806 19:43:00.677863 30328 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bee81cfbf57409982db0434f0beb764 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:43:00 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:55476
I20260806 19:43:00.679749 30188 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bee81cfbf57409982db0434f0beb764 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:43:00 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:40380
I20260806 19:43:00.693259 30630 tablet_replica.cc:330] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3: stopping tablet replica
I20260806 19:43:00.693521 30630 raft_consensus.cc:2229] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:43:00.693802 30630 raft_consensus.cc:2258] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:00.695156 30630 ts_tablet_manager.cc:1905] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:00.697468 30633 tablet_replica.cc:330] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149: stopping tablet replica
I20260806 19:43:00.697744 30633 raft_consensus.cc:2229] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:43:00.698526 30633 raft_consensus.cc:2258] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:00.699141 30631 tablet_replica.cc:330] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e: stopping tablet replica
I20260806 19:43:00.699375 30631 raft_consensus.cc:2229] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:43:00.699617 30631 raft_consensus.cc:2258] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:00.700795 30631 ts_tablet_manager.cc:1905] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:00.702728 30630 ts_tablet_manager.cc:1918] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:43:00.702875 30630 log.cc:1196] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/2bee81cfbf57409982db0434f0beb764
I20260806 19:43:00.703462 30630 ts_tablet_manager.cc:1939] T 2bee81cfbf57409982db0434f0beb764 P ec750bd3403b47cba074d1b138e717a3: Deleting consensus metadata
I20260806 19:43:00.704674 30036 catalog_manager.cc:4841] TS ec750bd3403b47cba074d1b138e717a3 (127.21.202.67:44865): tablet 2bee81cfbf57409982db0434f0beb764 (table test-table [id=a81418f109fd42d48871f55a2de7ed99]) successfully deleted
I20260806 19:43:00.707692 30633 ts_tablet_manager.cc:1905] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:00.708945 30631 ts_tablet_manager.cc:1918] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:43:00.709095 30631 log.cc:1196] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/2bee81cfbf57409982db0434f0beb764
I20260806 19:43:00.709532 30631 ts_tablet_manager.cc:1939] T 2bee81cfbf57409982db0434f0beb764 P 495b6134be8e4b7db72d06fe0900920e: Deleting consensus metadata
I20260806 19:43:00.710788 30036 catalog_manager.cc:4841] TS 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377): tablet 2bee81cfbf57409982db0434f0beb764 (table test-table [id=a81418f109fd42d48871f55a2de7ed99]) successfully deleted
I20260806 19:43:00.715883 30633 ts_tablet_manager.cc:1918] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:43:00.716046 30633 log.cc:1196] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/2bee81cfbf57409982db0434f0beb764
I20260806 19:43:00.716549 30633 ts_tablet_manager.cc:1939] T 2bee81cfbf57409982db0434f0beb764 P cc7f144fed494853a610eac95b529149: Deleting consensus metadata
I20260806 19:43:00.717818 30036 catalog_manager.cc:4841] TS cc7f144fed494853a610eac95b529149 (127.21.202.65:37773): tablet 2bee81cfbf57409982db0434f0beb764 (table test-table [id=a81418f109fd42d48871f55a2de7ed99]) successfully deleted
Aug 06 19:43:00 dist-test-slave-77v9 krb5kdc[30005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 19:43:00 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:00 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 19:43:00.757040 30606 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:00.745254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47760 (local address 127.21.202.126:46343)
0806 19:43:00.745488 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:00.745498 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:00.745904 (+ 406us) server_negotiation.cc:407] Connection header received
0806 19:43:00.746146 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:00.746151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:00.746242 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:00.746375 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:00.747857 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.748693 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.750200 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.750678 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.750841 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.756728 (+ 5887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47760: BlockingRecv error: recv got EOF from 127.0.0.1:47760 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 19:43:00 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM
I20260806 19:43:00.783977 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.761685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47766 (local address 127.21.202.126:46343)
0806 19:43:00.761864 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:00.761871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:00.765467 (+ 3596us) server_negotiation.cc:407] Connection header received
0806 19:43:00.765692 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:00.765698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:00.765800 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:00.765940 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:00.767550 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.768784 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.770711 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.771406 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.771568 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.776723 (+ 5155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:00.776730 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:00.776735 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:00.776787 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:00.780503 (+ 3716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:00.781519 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:00.781526 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:00.781532 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:00.781691 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:00.782156 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:00.782163 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:00.782167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:00.782627 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:00.782912 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:00.783322 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 19:43:00.783604 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 19:43:00.788496 30049 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:43:00.789287 30049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:43:00.797446 30188 tablet_service.cc:1467] Processing CreateTablet for tablet 99ab0d5fe8fa4906aa163e9b6bf06c07 (DEFAULT_TABLE table=test-table [id=f3b3ac7ae7c14c8989981884e5960006]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:43:00.797885 30328 tablet_service.cc:1467] Processing CreateTablet for tablet 99ab0d5fe8fa4906aa163e9b6bf06c07 (DEFAULT_TABLE table=test-table [id=f3b3ac7ae7c14c8989981884e5960006]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:43:00.798051 30188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ab0d5fe8fa4906aa163e9b6bf06c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:00.798141 30468 tablet_service.cc:1467] Processing CreateTablet for tablet 99ab0d5fe8fa4906aa163e9b6bf06c07 (DEFAULT_TABLE table=test-table [id=f3b3ac7ae7c14c8989981884e5960006]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:43:00.798341 30328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ab0d5fe8fa4906aa163e9b6bf06c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:00.798555 30468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ab0d5fe8fa4906aa163e9b6bf06c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:00.805406 30567 tablet_bootstrap.cc:492] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3: Bootstrap starting.
I20260806 19:43:00.806218 30646 tablet_bootstrap.cc:492] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e: Bootstrap starting.
I20260806 19:43:00.807246 30567 tablet_bootstrap.cc:654] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:00.807934 30646 tablet_bootstrap.cc:654] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:00.810546 30567 tablet_bootstrap.cc:492] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3: No bootstrap required, opened a new log
I20260806 19:43:00.810741 30567 ts_tablet_manager.cc:1397] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 19:43:00.811820 30567 raft_consensus.cc:348] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.812163 30567 raft_consensus.cc:374] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:00.812278 30567 raft_consensus.cc:729] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec750bd3403b47cba074d1b138e717a3, State: Initialized, Role: FOLLOWER
I20260806 19:43:00.812578 30567 consensus_queue.cc:260] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.815075 30567 ts_tablet_manager.cc:1428] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 19:43:00.816497 30646 tablet_bootstrap.cc:492] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e: No bootstrap required, opened a new log
I20260806 19:43:00.816743 30646 ts_tablet_manager.cc:1397] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260806 19:43:00.817919 30646 raft_consensus.cc:348] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.818277 30646 raft_consensus.cc:374] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:00.818420 30646 raft_consensus.cc:729] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 495b6134be8e4b7db72d06fe0900920e, State: Initialized, Role: FOLLOWER
I20260806 19:43:00.818804 30646 consensus_queue.cc:260] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.820588 30646 ts_tablet_manager.cc:1428] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260806 19:43:00.823768 30649 tablet_bootstrap.cc:492] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149: Bootstrap starting.
I20260806 19:43:00.826310 30649 tablet_bootstrap.cc:654] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:00.830422 30649 tablet_bootstrap.cc:492] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149: No bootstrap required, opened a new log
I20260806 19:43:00.830614 30649 ts_tablet_manager.cc:1397] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 19:43:00.831573 30649 raft_consensus.cc:348] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.831848 30649 raft_consensus.cc:374] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:00.831956 30649 raft_consensus.cc:729] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc7f144fed494853a610eac95b529149, State: Initialized, Role: FOLLOWER
I20260806 19:43:00.832201 30649 consensus_queue.cc:260] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.832796 30649 ts_tablet_manager.cc:1428] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 19:43:00.864286 30574 raft_consensus.cc:482] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:43:00.864593 30574 raft_consensus.cc:504] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.866272 30574 leader_election.cc:288] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc7f144fed494853a610eac95b529149 (127.21.202.65:37773), 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377)
I20260806 19:43:00.874946 30615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.866845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:42895 (local address 127.21.202.65:37773)
0806 19:43:00.867078 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:00.867087 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:00.867242 (+ 155us) server_negotiation.cc:407] Connection header received
0806 19:43:00.867449 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:00.867455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:00.867535 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:00.867636 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:43:00.868392 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.869600 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.872191 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.873615 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.873783 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.873862 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:00.874482 (+ 620us) server_negotiation.cc:299] Negotiation successful
0806 19:43:00.874634 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 19:43:00.875734 30651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.866437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37773 (local address 127.21.202.67:42895)
0806 19:43:00.867108 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:00.867130 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:00.867282 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:00.867843 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:00.867848 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:00.867877 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:43:00.868173 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.868188 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.869809 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.869817 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.871964 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.871980 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.874045 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.874052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.874263 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.874316 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 19:43:00.875216 (+ 900us) client_negotiation.cc:240] Negotiation successful
0806 19:43:00.875387 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":560,"threads_started":1}
I20260806 19:43:00.877741 30209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ab0d5fe8fa4906aa163e9b6bf06c07" candidate_uuid: "ec750bd3403b47cba074d1b138e717a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc7f144fed494853a610eac95b529149" is_pre_election: true
I20260806 19:43:00.878051 30209 raft_consensus.cc:2454] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec750bd3403b47cba074d1b138e717a3 in term 0.
I20260806 19:43:00.878600 30419 leader_election.cc:302] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc7f144fed494853a610eac95b529149, ec750bd3403b47cba074d1b138e717a3; no voters:
I20260806 19:43:00.879016 30574 raft_consensus.cc:2781] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:43:00.879150 30574 raft_consensus.cc:482] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:43:00.879253 30574 raft_consensus.cc:3037] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:00.882375 30574 raft_consensus.cc:504] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.883109 30574 leader_election.cc:288] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [CANDIDATE]: Term 1 election: Requested vote from peers cc7f144fed494853a610eac95b529149 (127.21.202.65:37773), 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377)
I20260806 19:43:00.883697 30651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.867468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:42377 (local address 127.21.202.67:46239)
0806 19:43:00.876007 (+ 8539us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:00.876029 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:00.876137 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:00.877443 (+ 1306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:00.877447 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:00.877477 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:43:00.877643 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.877652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.879066 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.879071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.881094 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:00.881116 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.882862 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.882868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:00.883090 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.883131 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 19:43:00.883231 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 19:43:00.883373 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8415,"thread_start_us":9508,"threads_started":1}
I20260806 19:43:00.883698 30209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ab0d5fe8fa4906aa163e9b6bf06c07" candidate_uuid: "ec750bd3403b47cba074d1b138e717a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc7f144fed494853a610eac95b529149"
I20260806 19:43:00.883978 30209 raft_consensus.cc:3037] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:00.884649 30653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.867690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:46239 (local address 127.21.202.66:42377)
0806 19:43:00.877090 (+ 9400us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:00.877097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:00.877117 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:43:00.877191 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:00.877197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:00.877284 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:00.877427 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:43:00.877803 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.878931 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.881316 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.882682 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.882901 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.882984 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:00.883700 (+ 716us) server_negotiation.cc:299] Negotiation successful
0806 19:43:00.884268 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9237,"thread_start_us":9037,"threads_started":1}
I20260806 19:43:00.885300 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ab0d5fe8fa4906aa163e9b6bf06c07" candidate_uuid: "ec750bd3403b47cba074d1b138e717a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495b6134be8e4b7db72d06fe0900920e" is_pre_election: true
I20260806 19:43:00.885434 30352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ab0d5fe8fa4906aa163e9b6bf06c07" candidate_uuid: "ec750bd3403b47cba074d1b138e717a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495b6134be8e4b7db72d06fe0900920e"
I20260806 19:43:00.885577 30349 raft_consensus.cc:2454] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec750bd3403b47cba074d1b138e717a3 in term 0.
I20260806 19:43:00.887036 30209 raft_consensus.cc:2454] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec750bd3403b47cba074d1b138e717a3 in term 1.
I20260806 19:43:00.887478 30419 leader_election.cc:302] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cc7f144fed494853a610eac95b529149, ec750bd3403b47cba074d1b138e717a3; no voters: 495b6134be8e4b7db72d06fe0900920e
I20260806 19:43:00.887884 30574 raft_consensus.cc:2781] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:43:00.888413 30574 raft_consensus.cc:686] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [term 1 LEADER]: Becoming Leader. State: Replica: ec750bd3403b47cba074d1b138e717a3, State: Running, Role: LEADER
I20260806 19:43:00.888850 30574 consensus_queue.cc:237] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:00.892884 30051 catalog_manager.cc:5495] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec750bd3403b47cba074d1b138e717a3 (127.21.202.67). New cstate: current_term: 1 leader_uuid: "ec750bd3403b47cba074d1b138e717a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } health_report { overall_health: HEALTHY } } }
I20260806 19:43:00.927793 30657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:00.918784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57082 (local address 127.21.202.67:44865)
0806 19:43:00.921347 (+ 2563us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:00.921353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:00.921380 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:43:00.921488 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:00.921493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:00.921615 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:00.921803 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:43:00.922220 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.923453 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.924788 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:00.925531 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:00.925665 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:00.926154 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:43:00.926971 (+ 817us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:43:00.927123 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:00.927308 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 19:43:00.927401 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2390,"thread_start_us":2226,"threads_started":1}
I20260806 19:43:00.931618 30352 raft_consensus.cc:3037] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:00.931829 30209 raft_consensus.cc:1261] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Refusing update from remote peer ec750bd3403b47cba074d1b138e717a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:43:00.932495 30654 consensus_queue.cc:1035] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:43:00.934871 30352 raft_consensus.cc:1261] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Refusing update from remote peer ec750bd3403b47cba074d1b138e717a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:43:00.935515 30654 consensus_queue.cc:1035] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:43:00.952538 30051 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:43:00.953424 30051 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:43:00.957370 30051 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61: Sending DeleteTablet for 3 replicas of tablet 99ab0d5fe8fa4906aa163e9b6bf06c07
I20260806 19:43:00.958185 30188 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ab0d5fe8fa4906aa163e9b6bf06c07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:43:00 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:40380
I20260806 19:43:00.958484 30633 tablet_replica.cc:330] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149: stopping tablet replica
I20260806 19:43:00.958679 30633 raft_consensus.cc:2229] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:43:00.958899 30633 raft_consensus.cc:2258] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:00.959846 30328 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ab0d5fe8fa4906aa163e9b6bf06c07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:43:00 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:55476
I20260806 19:43:00.960166 30631 tablet_replica.cc:330] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e: stopping tablet replica
I20260806 19:43:00.960397 30631 raft_consensus.cc:2229] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:43:00.960501 30328 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ab0d5fe8fa4906aa163e9b6bf06c07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:43:00 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:55476
I20260806 19:43:00.960664 30631 raft_consensus.cc:2258] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:00.961009 30468 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ab0d5fe8fa4906aa163e9b6bf06c07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:43:00 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:57080
I20260806 19:43:00.961316 30630 tablet_replica.cc:330] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3: stopping tablet replica
I20260806 19:43:00.961544 30630 raft_consensus.cc:2229] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:43:00.961849 30631 ts_tablet_manager.cc:1905] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:00.961946 30633 ts_tablet_manager.cc:1905] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:00.962078 30630 raft_consensus.cc:2258] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:00.963474 30630 ts_tablet_manager.cc:1905] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:00.968261 30051 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:43:00.969012 30051 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:43:00.974653 30631 ts_tablet_manager.cc:1918] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:43:00.974653 30630 ts_tablet_manager.cc:1918] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:43:00.974839 30631 log.cc:1196] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/99ab0d5fe8fa4906aa163e9b6bf06c07
I20260806 19:43:00.974839 30630 log.cc:1196] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/99ab0d5fe8fa4906aa163e9b6bf06c07
I20260806 19:43:00.975332 30630 ts_tablet_manager.cc:1939] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P ec750bd3403b47cba074d1b138e717a3: Deleting consensus metadata
I20260806 19:43:00.975360 30631 ts_tablet_manager.cc:1939] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P 495b6134be8e4b7db72d06fe0900920e: Deleting consensus metadata
I20260806 19:43:00.976621 30036 catalog_manager.cc:4841] TS 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377): tablet 99ab0d5fe8fa4906aa163e9b6bf06c07 (table test-table [id=f3b3ac7ae7c14c8989981884e5960006]) successfully deleted
I20260806 19:43:00.978842 30328 tablet_service.cc:1467] Processing CreateTablet for tablet d6ef2bb2e8fc4515a87dae7debbd775f (DEFAULT_TABLE table=test-table [id=58dacfbec35a40beac8e62bb10daaa20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:43:00.978992 30188 tablet_service.cc:1467] Processing CreateTablet for tablet d6ef2bb2e8fc4515a87dae7debbd775f (DEFAULT_TABLE table=test-table [id=58dacfbec35a40beac8e62bb10daaa20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:43:00.979285 30328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ef2bb2e8fc4515a87dae7debbd775f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:00.979470 30188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ef2bb2e8fc4515a87dae7debbd775f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:00.986007 30646 tablet_bootstrap.cc:492] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e: Bootstrap starting.
I20260806 19:43:00.985920 30468 tablet_service.cc:1467] Processing CreateTablet for tablet d6ef2bb2e8fc4515a87dae7debbd775f (DEFAULT_TABLE table=test-table [id=58dacfbec35a40beac8e62bb10daaa20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:43:00.986436 30468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ef2bb2e8fc4515a87dae7debbd775f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:00.988047 30646 tablet_bootstrap.cc:654] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:00.989432 30036 catalog_manager.cc:4841] TS ec750bd3403b47cba074d1b138e717a3 (127.21.202.67:44865): tablet 99ab0d5fe8fa4906aa163e9b6bf06c07 (table test-table [id=f3b3ac7ae7c14c8989981884e5960006]) successfully deleted
I20260806 19:43:00.997236 30649 tablet_bootstrap.cc:492] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149: Bootstrap starting.
W20260806 19:43:00.997893 30036 catalog_manager.cc:4808] TS 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377): delete failed for tablet 99ab0d5fe8fa4906aa163e9b6bf06c07 because the tablet was not found. No further retry: Not found: Tablet not found: 99ab0d5fe8fa4906aa163e9b6bf06c07
I20260806 19:43:00.999151 30649 tablet_bootstrap.cc:654] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:01.000948 30567 tablet_bootstrap.cc:492] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3: Bootstrap starting.
I20260806 19:43:01.003331 30567 tablet_bootstrap.cc:654] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:01.009099 30649 tablet_bootstrap.cc:492] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149: No bootstrap required, opened a new log
I20260806 19:43:01.009292 30649 ts_tablet_manager.cc:1397] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.005s
I20260806 19:43:01.009393 30567 tablet_bootstrap.cc:492] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3: No bootstrap required, opened a new log
I20260806 19:43:01.009562 30567 ts_tablet_manager.cc:1397] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260806 19:43:01.010191 30633 ts_tablet_manager.cc:1918] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:43:01.010178 30649 raft_consensus.cc:348] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:01.010367 30567 raft_consensus.cc:348] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:01.010521 30649 raft_consensus.cc:374] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:01.010556 30633 log.cc:1196] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/99ab0d5fe8fa4906aa163e9b6bf06c07
I20260806 19:43:01.010633 30567 raft_consensus.cc:374] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:01.010715 30567 raft_consensus.cc:729] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec750bd3403b47cba074d1b138e717a3, State: Initialized, Role: FOLLOWER
I20260806 19:43:01.010984 30567 consensus_queue.cc:260] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:01.010615 30649 raft_consensus.cc:729] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc7f144fed494853a610eac95b529149, State: Initialized, Role: FOLLOWER
I20260806 19:43:01.012035 30649 consensus_queue.cc:260] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:01.013710 30649 ts_tablet_manager.cc:1428] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 19:43:01.014374 30633 ts_tablet_manager.cc:1939] T 99ab0d5fe8fa4906aa163e9b6bf06c07 P cc7f144fed494853a610eac95b529149: Deleting consensus metadata
I20260806 19:43:01.017481 30567 ts_tablet_manager.cc:1428] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260806 19:43:01.018345 30036 catalog_manager.cc:4841] TS cc7f144fed494853a610eac95b529149 (127.21.202.65:37773): tablet 99ab0d5fe8fa4906aa163e9b6bf06c07 (table test-table [id=f3b3ac7ae7c14c8989981884e5960006]) successfully deleted
I20260806 19:43:01.018887 30646 tablet_bootstrap.cc:492] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e: No bootstrap required, opened a new log
I20260806 19:43:01.019141 30646 ts_tablet_manager.cc:1397] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e: Time spent bootstrapping tablet: real 0.033s user 0.014s sys 0.005s
I20260806 19:43:01.020574 30646 raft_consensus.cc:348] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:01.021030 30646 raft_consensus.cc:374] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:01.021176 30646 raft_consensus.cc:729] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 495b6134be8e4b7db72d06fe0900920e, State: Initialized, Role: FOLLOWER
I20260806 19:43:01.021502 30646 consensus_queue.cc:260] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:01.022156 30646 ts_tablet_manager.cc:1428] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 19:43:01.137087 30572 raft_consensus.cc:482] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:43:01.137364 30572 raft_consensus.cc:504] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:01.138222 30572 leader_election.cc:288] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377), ec750bd3403b47cba074d1b138e717a3 (127.21.202.67:44865)
I20260806 19:43:01.138954 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ef2bb2e8fc4515a87dae7debbd775f" candidate_uuid: "cc7f144fed494853a610eac95b529149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495b6134be8e4b7db72d06fe0900920e" is_pre_election: true
I20260806 19:43:01.139250 30349 raft_consensus.cc:2454] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc7f144fed494853a610eac95b529149 in term 0.
I20260806 19:43:01.139271 30489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ef2bb2e8fc4515a87dae7debbd775f" candidate_uuid: "cc7f144fed494853a610eac95b529149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec750bd3403b47cba074d1b138e717a3" is_pre_election: true
I20260806 19:43:01.139597 30489 raft_consensus.cc:2454] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc7f144fed494853a610eac95b529149 in term 0.
I20260806 19:43:01.140183 30139 leader_election.cc:302] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc7f144fed494853a610eac95b529149, ec750bd3403b47cba074d1b138e717a3; no voters:
I20260806 19:43:01.140558 30572 raft_consensus.cc:2781] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:43:01.140678 30572 raft_consensus.cc:482] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:43:01.140801 30572 raft_consensus.cc:3037] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:01.144356 30572 raft_consensus.cc:504] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:01.145154 30572 leader_election.cc:288] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [CANDIDATE]: Term 1 election: Requested vote from peers 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377), ec750bd3403b47cba074d1b138e717a3 (127.21.202.67:44865)
I20260806 19:43:01.145838 30489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ef2bb2e8fc4515a87dae7debbd775f" candidate_uuid: "cc7f144fed494853a610eac95b529149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec750bd3403b47cba074d1b138e717a3"
I20260806 19:43:01.146090 30489 raft_consensus.cc:3037] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:01.146005 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ef2bb2e8fc4515a87dae7debbd775f" candidate_uuid: "cc7f144fed494853a610eac95b529149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495b6134be8e4b7db72d06fe0900920e"
I20260806 19:43:01.146242 30349 raft_consensus.cc:3037] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:01.149391 30349 raft_consensus.cc:2454] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc7f144fed494853a610eac95b529149 in term 1.
I20260806 19:43:01.149469 30489 raft_consensus.cc:2454] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc7f144fed494853a610eac95b529149 in term 1.
I20260806 19:43:01.149955 30139 leader_election.cc:302] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc7f144fed494853a610eac95b529149, ec750bd3403b47cba074d1b138e717a3; no voters:
I20260806 19:43:01.150303 30572 raft_consensus.cc:2781] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:43:01.150970 30572 raft_consensus.cc:686] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [term 1 LEADER]: Becoming Leader. State: Replica: cc7f144fed494853a610eac95b529149, State: Running, Role: LEADER
I20260806 19:43:01.151353 30572 consensus_queue.cc:237] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } }
I20260806 19:43:01.155000 30049 catalog_manager.cc:5495] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc7f144fed494853a610eac95b529149 (127.21.202.65). New cstate: current_term: 1 leader_uuid: "cc7f144fed494853a610eac95b529149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc7f144fed494853a610eac95b529149" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 } health_report { overall_health: UNKNOWN } } }
I20260806 19:43:01.201640 30615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:01.194933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40416 (local address 127.21.202.65:37773)
0806 19:43:01.195096 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:01.195102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:01.195201 (+ 99us) server_negotiation.cc:407] Connection header received
0806 19:43:01.195458 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:01.195465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:01.195557 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:01.195657 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:43:01.196102 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:01.197313 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:01.199358 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:01.200102 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:01.200251 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:01.200770 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:43:01.200931 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:43:01.201034 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:01.201210 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 19:43:01.201317 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 19:43:01.222282 30489 raft_consensus.cc:1261] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Refusing update from remote peer cc7f144fed494853a610eac95b529149: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:43:01.222503 30349 raft_consensus.cc:1261] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Refusing update from remote peer cc7f144fed494853a610eac95b529149: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:43:01.223449 30665 consensus_queue.cc:1035] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "495b6134be8e4b7db72d06fe0900920e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:43:01.223788 30668 consensus_queue.cc:1035] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec750bd3403b47cba074d1b138e717a3" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 44865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:43:01.237797 30582 mvcc.cc:205] Tried to move back new op lower bound from 7315641881446137856 to 7315641881199489024. Current Snapshot: MvccSnapshot[applied={T|T < 7315641881446137856}]
I20260806 19:43:01.301069 30051 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:43:01.301381 30051 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:43:01.305141 30051 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63bdaa10658146dab4b821f4dcb52f61: Sending DeleteTablet for 3 replicas of tablet d6ef2bb2e8fc4515a87dae7debbd775f
I20260806 19:43:01.306178 30468 tablet_service.cc:1514] Processing DeleteTablet for tablet d6ef2bb2e8fc4515a87dae7debbd775f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:43:01 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:57080
I20260806 19:43:01.306205 30328 tablet_service.cc:1514] Processing DeleteTablet for tablet d6ef2bb2e8fc4515a87dae7debbd775f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:43:01 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:55476
I20260806 19:43:01.306535 30630 tablet_replica.cc:330] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3: stopping tablet replica
I20260806 19:43:01.306538 30631 tablet_replica.cc:330] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e: stopping tablet replica
I20260806 19:43:01.306742 30631 raft_consensus.cc:2229] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:43:01.306754 30630 raft_consensus.cc:2229] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:43:01.306980 30630 raft_consensus.cc:2258] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:01.307001 30631 raft_consensus.cc:2258] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:01.307967 30189 tablet_service.cc:1514] Processing DeleteTablet for tablet d6ef2bb2e8fc4515a87dae7debbd775f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:43:01 UTC) from {username='oryx', principal='oryx/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:40380
I20260806 19:43:01.308247 30630 ts_tablet_manager.cc:1905] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:01.308292 30633 tablet_replica.cc:330] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149: stopping tablet replica
I20260806 19:43:01.308568 30633 raft_consensus.cc:2229] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:43:01.309763 30631 ts_tablet_manager.cc:1905] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:01.309778 30633 raft_consensus.cc:2258] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:01.313436 30633 ts_tablet_manager.cc:1905] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:01.316648 30631 ts_tablet_manager.cc:1918] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:43:01.316787 30631 log.cc:1196] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/d6ef2bb2e8fc4515a87dae7debbd775f
I20260806 19:43:01.317263 30631 ts_tablet_manager.cc:1939] T d6ef2bb2e8fc4515a87dae7debbd775f P 495b6134be8e4b7db72d06fe0900920e: Deleting consensus metadata
I20260806 19:43:01.317528 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 30122
I20260806 19:43:01.319114 30036 catalog_manager.cc:4841] TS 495b6134be8e4b7db72d06fe0900920e (127.21.202.66:42377): tablet d6ef2bb2e8fc4515a87dae7debbd775f (table test-table [id=58dacfbec35a40beac8e62bb10daaa20]) successfully deleted
I20260806 19:43:01.325430 30630 ts_tablet_manager.cc:1918] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:43:01.329670 30633 ts_tablet_manager.cc:1918] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:43:01.329932 30630 log.cc:1196] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/d6ef2bb2e8fc4515a87dae7debbd775f
I20260806 19:43:01.330116 30633 log.cc:1196] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/d6ef2bb2e8fc4515a87dae7debbd775f
I20260806 19:43:01.330694 30630 ts_tablet_manager.cc:1939] T d6ef2bb2e8fc4515a87dae7debbd775f P ec750bd3403b47cba074d1b138e717a3: Deleting consensus metadata
I20260806 19:43:01.331267 30633 ts_tablet_manager.cc:1939] T d6ef2bb2e8fc4515a87dae7debbd775f P cc7f144fed494853a610eac95b529149: Deleting consensus metadata
I20260806 19:43:01.332755 30036 catalog_manager.cc:4841] TS ec750bd3403b47cba074d1b138e717a3 (127.21.202.67:44865): tablet d6ef2bb2e8fc4515a87dae7debbd775f (table test-table [id=58dacfbec35a40beac8e62bb10daaa20]) successfully deleted
I20260806 19:43:01.333938 30036 catalog_manager.cc:4841] TS cc7f144fed494853a610eac95b529149 (127.21.202.65:37773): tablet d6ef2bb2e8fc4515a87dae7debbd775f (table test-table [id=58dacfbec35a40beac8e62bb10daaa20]) successfully deleted
Aug 06 19:43:01 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.202.65@KRBTEST.COM
I20260806 19:43:01.351536 30615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:01.319090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40418 (local address 127.21.202.65:37773)
0806 19:43:01.319323 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:01.319329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:01.330309 (+ 10980us) server_negotiation.cc:407] Connection header received
0806 19:43:01.330644 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:01.330651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:01.330761 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:01.330841 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:01.333461 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:01.334581 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:01.336494 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:01.337272 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:01.337418 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:01.343188 (+ 5770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:01.343208 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:01.343240 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:01.343286 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:01.347632 (+ 4346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:01.348605 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:01.348616 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:01.348620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:01.348839 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:01.349332 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:01.349337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:01.349341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:01.349852 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:01.350154 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:01.350603 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 19:43:01.350889 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 19:43:01.352275 30254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:01.526795 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 30122
I20260806 19:43:01.554556 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 30263
Aug 06 19:43:01 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.202.66@KRBTEST.COM
I20260806 19:43:01.571125 30674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:01.555451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55496 (local address 127.21.202.66:42377)
0806 19:43:01.555956 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:01.555960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:01.555978 (+ 18us) server_negotiation.cc:407] Connection header received
0806 19:43:01.556034 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:01.556037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:01.556116 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:01.556207 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:01.557623 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:01.558433 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:01.559856 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:01.560373 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:01.560479 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:01.564994 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:01.565010 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:01.565026 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:01.565068 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:01.568140 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:01.569003 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:01.569011 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:01.569013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:01.569138 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:01.569564 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:01.569568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:01.569570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:01.569954 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:01.570163 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:01.570537 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 19:43:01.570719 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":282,"threads_started":1}
I20260806 19:43:01.571852 30393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:01.700654 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 30263
I20260806 19:43:01.725675 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 30402
Aug 06 19:43:01 dist-test-slave-77v9 krb5kdc[30005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.202.67@KRBTEST.COM
I20260806 19:43:01.742285 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:01.726504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57084 (local address 127.21.202.67:44865)
0806 19:43:01.727008 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:01.727013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:01.727031 (+ 18us) server_negotiation.cc:407] Connection header received
0806 19:43:01.727092 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:01.727096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:01.727162 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:01.727278 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:01.728552 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:01.729388 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:01.730652 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:01.731149 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:01.731258 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:01.736032 (+ 4774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:01.736048 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:01.736065 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:01.736102 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:01.739300 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:01.740082 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:01.740092 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:01.740095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:01.740312 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:01.740921 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:01.740925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:01.740927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:01.741290 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:01.741561 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:01.741801 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 19:43:01.742000 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":283,"threads_started":1}
I20260806 19:43:01.743018 30536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:01.872742 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 30402
I20260806 19:43:01.898204 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 30019
I20260806 19:43:01.899466 30091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:02.020735 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 30019
2026-08-06T19:43:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6158 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:43:02 dist-test-slave-77v9 krb5kdc[30681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:43:02 dist-test-slave-77v9 krb5kdc[30681](info): set up 2 sockets
Aug 06 19:43:02 dist-test-slave-77v9 krb5kdc[30681](info): commencing operation
krb5kdc: starting...
W20260806 19:43:03.174170 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.008s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:43:03 dist-test-slave-77v9 krb5kdc[30681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:43:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:43:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:43:03.465898 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42797
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:42797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:43:03.818259 30694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:03.818562 30694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:03.818735 30694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:03.826181 30694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:43:03.826292 30694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:03.826400 30694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:03.826529 30694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:43:03.826612 30694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:43:03.836951 30694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:42797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:03.838322 30694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:03.839663 30694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:03.853139 30699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:03.853173 30702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:03.853209 30700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:03.854012 30694 server_base.cc:1034] running on GCE node
I20260806 19:43:03.854838 30694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:03.856345 30694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:03.857600 30694 hybrid_clock.cc:648] HybridClock initialized: now 1786045383857555 us; error 65 us; skew 500 ppm
I20260806 19:43:03.858000 30694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T19:43:03Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1907 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:43:04 dist-test-slave-77v9 krb5kdc[30708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:43:04 dist-test-slave-77v9 krb5kdc[30708](info): set up 2 sockets
Aug 06 19:43:04 dist-test-slave-77v9 krb5kdc[30708](info): commencing operation
krb5kdc: starting...
W20260806 19:43:05.159186 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.006s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:43:05 dist-test-slave-77v9 krb5kdc[30708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:43:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:43:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:43:05.440882 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42403
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:42403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:43:05.789974 30722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:05.790277 30722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:05.790462 30722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:05.797798 30722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:43:05.797915 30722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:05.798018 30722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:05.798128 30722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:43:05.798211 30722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:43:05.808259 30722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:42403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:05.809705 30722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:05.811086 30722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:05.823786 30728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:05.823853 30727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:05.823760 30731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:05.824620 30722 server_base.cc:1034] running on GCE node
I20260806 19:43:05.825512 30722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:05.827144 30722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:05.828379 30722 hybrid_clock.cc:648] HybridClock initialized: now 1786045385828338 us; error 31 us; skew 500 ppm
I20260806 19:43:05.828711 30722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:43:05 dist-test-slave-77v9 krb5kdc[30708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045385, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:43:05.837771 30722 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:43:05.840117 30722 webserver.cc:460] Webserver started at http://127.21.202.126:36251/ using document root <none> and password file <none>
I20260806 19:43:05.841049 30722 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:05.841199 30722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:05.841657 30722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:05.845247 30722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "eef092edec974e4eac904a8ce63c984a"
format_stamp: "Formatted at 2026-08-06 19:43:05 on dist-test-slave-77v9"
server_key: "208dd554d7be4cef5f97bee166aab170"
server_key_iv: "b77fb13f79865264178d965eccf1fcc5"
server_key_version: "encryptionkey@0"
I20260806 19:43:05.846141 30722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "eef092edec974e4eac904a8ce63c984a"
format_stamp: "Formatted at 2026-08-06 19:43:05 on dist-test-slave-77v9"
server_key: "208dd554d7be4cef5f97bee166aab170"
server_key_iv: "b77fb13f79865264178d965eccf1fcc5"
server_key_version: "encryptionkey@0"
I20260806 19:43:05.852528 30722 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 19:43:05.856341 30738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:05.857890 30722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:43:05.858088 30722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "eef092edec974e4eac904a8ce63c984a"
format_stamp: "Formatted at 2026-08-06 19:43:05 on dist-test-slave-77v9"
server_key: "208dd554d7be4cef5f97bee166aab170"
server_key_iv: "b77fb13f79865264178d965eccf1fcc5"
server_key_version: "encryptionkey@0"
I20260806 19:43:05.858278 30722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:05.871106 30722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:05.875765 30722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:05.876101 30722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:05.896100 30722 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:42403
I20260806 19:43:05.896121 30789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:42403 every 8 connection(s)
I20260806 19:43:05.897626 30722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:43:05.898518 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 30722
I20260806 19:43:05.898782 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:43:05.899482 22313 external_mini_cluster.cc:1428] Setting key 0aa7ff7efd9466c575bd94cb4c809b5a
I20260806 19:43:05.902205 30790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:05.913458 30790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a: Bootstrap starting.
Aug 06 19:43:05 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:43:05.917353 30790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:05.918509 30790 log.cc:824] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:43:05.921949 30790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a: No bootstrap required, opened a new log
I20260806 19:43:05.923629 30793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:05.901555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36502 (local address 127.21.202.126:42403)
0806 19:43:05.902474 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:05.902490 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:05.902539 (+ 49us) server_negotiation.cc:407] Connection header received
0806 19:43:05.904310 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:05.904333 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:05.904744 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:05.905252 (+ 508us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:05.906958 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:05.908360 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:05.909816 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:05.910584 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:05.910877 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:05.915519 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:05.915528 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:05.915542 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:05.915587 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:05.919367 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:05.920169 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:05.920176 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:05.920181 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:05.920345 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:05.920770 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:05.920774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:05.920776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:05.921385 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:05.921604 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:05.922196 (+ 592us) server_negotiation.cc:299] Negotiation successful
0806 19:43:05.922635 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":390,"threads_started":1}
I20260806 19:43:05.930266 30790 raft_consensus.cc:348] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef092edec974e4eac904a8ce63c984a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42403 } }
I20260806 19:43:05.930675 30790 raft_consensus.cc:374] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:05.930814 30790 raft_consensus.cc:729] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef092edec974e4eac904a8ce63c984a, State: Initialized, Role: FOLLOWER
I20260806 19:43:05.931414 30790 consensus_queue.cc:260] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef092edec974e4eac904a8ce63c984a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42403 } }
I20260806 19:43:05.931676 30790 raft_consensus.cc:388] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:43:05.931820 30790 raft_consensus.cc:482] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:43:05.932004 30790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:05.934242 30790 raft_consensus.cc:504] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef092edec974e4eac904a8ce63c984a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42403 } }
I20260806 19:43:05.934748 30790 leader_election.cc:302] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eef092edec974e4eac904a8ce63c984a; no voters:
I20260806 19:43:05.935350 30790 leader_election.cc:288] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:43:05.935509 30795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:43:05.936203 30795 raft_consensus.cc:686] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 1 LEADER]: Becoming Leader. State: Replica: eef092edec974e4eac904a8ce63c984a, State: Running, Role: LEADER
I20260806 19:43:05.936745 30795 consensus_queue.cc:237] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef092edec974e4eac904a8ce63c984a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42403 } }
I20260806 19:43:05.937529 30790 sys_catalog.cc:564] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:43:05.939801 30797 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [sys.catalog]: SysCatalogTable state changed. Reason: New leader eef092edec974e4eac904a8ce63c984a. Latest consensus state: current_term: 1 leader_uuid: "eef092edec974e4eac904a8ce63c984a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef092edec974e4eac904a8ce63c984a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42403 } } }
I20260806 19:43:05.940297 30797 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [sys.catalog]: This master's current role is: LEADER
I20260806 19:43:05.940994 30796 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eef092edec974e4eac904a8ce63c984a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef092edec974e4eac904a8ce63c984a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42403 } } }
I20260806 19:43:05.941213 30796 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [sys.catalog]: This master's current role is: LEADER
I20260806 19:43:05.941421 30804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:43:05.947886 30804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:43:05.958370 30804 catalog_manager.cc:1347] Generated new cluster ID: 659976977e4f4f86b7461a0ab9a91192
I20260806 19:43:05.958559 30804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:43:05.970808 30804 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:43:05.972220 30804 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:43:05.980492 30804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a: Generated new TSK 0
I20260806 19:43:05.981495 30804 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:43:06.053814 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42403
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:43:06.396025 30816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:06.396384 30816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:06.396567 30816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:06.404992 30816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:06.405174 30816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:06.405386 30816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:43:06.415863 30816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:06.417444 30816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:06.419072 30816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:06.433576 30824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:06.433547 30821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:06.433507 30822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:06.434387 30816 server_base.cc:1034] running on GCE node
I20260806 19:43:06.435066 30816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:06.436058 30816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:06.437355 30816 hybrid_clock.cc:648] HybridClock initialized: now 1786045386437248 us; error 66 us; skew 500 ppm
I20260806 19:43:06.437762 30816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:43:06 dist-test-slave-77v9 krb5kdc[30708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045386, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:43:06.441330 30816 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:43:06.443002 30816 webserver.cc:460] Webserver started at http://127.21.202.65:34825/ using document root <none> and password file <none>
I20260806 19:43:06.443733 30816 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:06.443838 30816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:06.444110 30816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:06.447175 30816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "c332c665fe094e2c9a4ad57fbafaff4c"
format_stamp: "Formatted at 2026-08-06 19:43:06 on dist-test-slave-77v9"
server_key: "13d6f6adf312a1dd69c494e2fe5c9b7e"
server_key_iv: "9b430ba81abbed7d7235b6e518f1fcca"
server_key_version: "encryptionkey@0"
I20260806 19:43:06.447935 30816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "c332c665fe094e2c9a4ad57fbafaff4c"
format_stamp: "Formatted at 2026-08-06 19:43:06 on dist-test-slave-77v9"
server_key: "13d6f6adf312a1dd69c494e2fe5c9b7e"
server_key_iv: "9b430ba81abbed7d7235b6e518f1fcca"
server_key_version: "encryptionkey@0"
I20260806 19:43:06.453421 30816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 19:43:06.457048 30831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:06.458887 30816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:43:06.459057 30816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "c332c665fe094e2c9a4ad57fbafaff4c"
format_stamp: "Formatted at 2026-08-06 19:43:06 on dist-test-slave-77v9"
server_key: "13d6f6adf312a1dd69c494e2fe5c9b7e"
server_key_iv: "9b430ba81abbed7d7235b6e518f1fcca"
server_key_version: "encryptionkey@0"
I20260806 19:43:06.459268 30816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:06.484707 30816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:06.489542 30816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:06.489810 30816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:06.490805 30816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:06.492172 30816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:06.492250 30816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:06.492336 30816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:06.492379 30816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:06.529493 30816 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:39763
I20260806 19:43:06.529536 30945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:39763 every 8 connection(s)
I20260806 19:43:06.530988 30816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:43:06.533975 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 30816
I20260806 19:43:06.534227 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:43:06.535126 22313 external_mini_cluster.cc:1428] Setting key 39fcdc87d9388bf743eebec8d476b154
Aug 06 19:43:06 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045386, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:43:06.557286 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:06.533956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:49129 (local address 127.21.202.126:42403)
0806 19:43:06.534577 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:06.534583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:06.535528 (+ 945us) server_negotiation.cc:407] Connection header received
0806 19:43:06.537982 (+ 2454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:06.537993 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:06.538166 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:06.538278 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:06.540911 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:06.542108 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:06.544986 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:06.545451 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:06.545573 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:06.550803 (+ 5230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:06.550809 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:06.550812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:06.550850 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:06.553637 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:06.554688 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:06.554692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:06.554694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:06.554822 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:06.555333 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:06.555336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:06.555338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:06.555629 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:06.555743 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:06.556747 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0806 19:43:06.557012 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":249,"threads_started":1}
I20260806 19:43:06.558823 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:06.534467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:49129)
0806 19:43:06.535329 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:06.535380 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:06.537615 (+ 2235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:06.538612 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:06.538626 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:06.539512 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:06.540407 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:06.540438 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:06.542351 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:06.542360 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:06.544764 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:06.544788 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:06.545741 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:06.545752 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:06.546140 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:06.547121 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:06.547162 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:06.550494 (+ 3332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:43:06.553817 (+ 3323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:06.553829 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:06.553852 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:06.554541 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:06.554939 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:06.554946 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:06.554952 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:06.555215 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:06.555772 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:43:06.555783 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:43:06.556293 (+ 510us) client_negotiation.cc:769] Sending connection context
0806 19:43:06.556623 (+ 330us) client_negotiation.cc:240] Negotiation successful
0806 19:43:06.557301 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":294,"threads_started":1}
I20260806 19:43:06.560498 30946 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42403
I20260806 19:43:06.560878 30946 heartbeater.cc:464] Registering TS with master...
I20260806 19:43:06.561869 30946 heartbeater.cc:511] Master 127.21.202.126:42403 requested a full tablet report, sending...
I20260806 19:43:06.564541 30755 ts_manager.cc:194] Registered new tserver with Master: c332c665fe094e2c9a4ad57fbafaff4c (127.21.202.65:39763)
I20260806 19:43:06.566263 30755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:49129
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:43:06.605839 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42403
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:43:06.956611 30952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:06.956956 30952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:06.957116 30952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:06.964483 30952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:06.964632 30952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:06.964843 30952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:43:06.974620 30952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:06.976032 30952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:06.977437 30952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:06.990794 30960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:06.990891 30958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:06.991057 30957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:06.994457 30952 server_base.cc:1034] running on GCE node
I20260806 19:43:06.995128 30952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:06.996192 30952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:06.997426 30952 hybrid_clock.cc:648] HybridClock initialized: now 1786045386997381 us; error 37 us; skew 500 ppm
I20260806 19:43:06.997753 30952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:43:06 dist-test-slave-77v9 krb5kdc[30708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045386, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:43:07.001044 30952 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:43:07.002600 30952 webserver.cc:460] Webserver started at http://127.21.202.66:42405/ using document root <none> and password file <none>
I20260806 19:43:07.003378 30952 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:07.003484 30952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:07.003738 30952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:07.006763 30952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "1d4f42c63c004d6c831b2880f35858dc"
format_stamp: "Formatted at 2026-08-06 19:43:06 on dist-test-slave-77v9"
server_key: "c630dc78dfd2b5af50fb48ee4b63d637"
server_key_iv: "b5b11715ec12f014660fad54aa5d957a"
server_key_version: "encryptionkey@0"
I20260806 19:43:07.007536 30952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "1d4f42c63c004d6c831b2880f35858dc"
format_stamp: "Formatted at 2026-08-06 19:43:06 on dist-test-slave-77v9"
server_key: "c630dc78dfd2b5af50fb48ee4b63d637"
server_key_iv: "b5b11715ec12f014660fad54aa5d957a"
server_key_version: "encryptionkey@0"
I20260806 19:43:07.012744 30952 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 19:43:07.016503 30967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:07.017995 30952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:43:07.018181 30952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "1d4f42c63c004d6c831b2880f35858dc"
format_stamp: "Formatted at 2026-08-06 19:43:06 on dist-test-slave-77v9"
server_key: "c630dc78dfd2b5af50fb48ee4b63d637"
server_key_iv: "b5b11715ec12f014660fad54aa5d957a"
server_key_version: "encryptionkey@0"
I20260806 19:43:07.018411 30952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:07.035156 30952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:07.039724 30952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:07.040037 30952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:07.041112 30952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:07.042608 30952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:07.042683 30952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:07.042773 30952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:07.042845 30952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:07.077894 30952 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:36733
I20260806 19:43:07.077950 31081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:36733 every 8 connection(s)
I20260806 19:43:07.079226 30952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:43:07.084668 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 30952
I20260806 19:43:07.084918 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:43:07.085465 22313 external_mini_cluster.cc:1428] Setting key ec1af652f5f89f857ad162c46149fc1d
Aug 06 19:43:07 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045386, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:43:07.102144 31084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:07.081914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:52613 (local address 127.21.202.126:42403)
0806 19:43:07.082470 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:07.082475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:07.083349 (+ 874us) server_negotiation.cc:407] Connection header received
0806 19:43:07.085722 (+ 2373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:07.085727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:07.085816 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:07.085916 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:07.087790 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:07.088530 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:07.091148 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:07.091596 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:07.091702 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:07.096112 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:07.096117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:07.096120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:07.096153 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:07.098877 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:07.099667 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:07.099677 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:07.099679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:07.099814 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:07.100273 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:07.100283 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:07.100286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:07.100560 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:07.100724 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:07.101520 (+ 796us) server_negotiation.cc:299] Negotiation successful
0806 19:43:07.101755 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":293,"threads_started":1}
I20260806 19:43:07.103156 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:07.082435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:52613)
0806 19:43:07.083193 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:07.083229 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:07.085408 (+ 2179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:07.086197 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:07.086205 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:07.086884 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:07.087591 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:07.087618 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:07.088713 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:07.088721 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:07.090981 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:07.090999 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:07.091839 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:07.091849 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:07.092192 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:07.092973 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:07.093008 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:07.095839 (+ 2831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:43:07.099050 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:07.099060 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:07.099077 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:07.099526 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:07.099933 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:07.099941 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:07.099944 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:07.100147 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:07.100698 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:43:07.100706 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:43:07.101139 (+ 433us) client_negotiation.cc:769] Sending connection context
0806 19:43:07.101418 (+ 279us) client_negotiation.cc:240] Negotiation successful
0806 19:43:07.101975 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":281,"threads_started":1}
I20260806 19:43:07.104691 31082 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42403
I20260806 19:43:07.105088 31082 heartbeater.cc:464] Registering TS with master...
I20260806 19:43:07.106004 31082 heartbeater.cc:511] Master 127.21.202.126:42403 requested a full tablet report, sending...
I20260806 19:43:07.107846 30755 ts_manager.cc:194] Registered new tserver with Master: 1d4f42c63c004d6c831b2880f35858dc (127.21.202.66:36733)
I20260806 19:43:07.109268 30755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:52613
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:43:07.149729 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42403
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:43:07.493767 31088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:07.494071 31088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:07.494217 31088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:07.501660 31088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:07.501783 31088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:07.501981 31088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:43:07.512722 31088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:07.514293 31088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:07.515630 31088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:07.531435 31093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:07.531469 31094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:07.534189 31096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:07.534631 31088 server_base.cc:1034] running on GCE node
I20260806 19:43:07.535322 31088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:07.536408 31088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:07.537622 31088 hybrid_clock.cc:648] HybridClock initialized: now 1786045387537571 us; error 37 us; skew 500 ppm
I20260806 19:43:07.537976 31088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:43:07 dist-test-slave-77v9 krb5kdc[30708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045387, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:43:07.541544 31088 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:43:07.543294 31088 webserver.cc:460] Webserver started at http://127.21.202.67:39783/ using document root <none> and password file <none>
I20260806 19:43:07.544029 31088 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:07.544173 31088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:07.544497 31088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:07.547539 31088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "6ce65b879ea64f6799cb460d6234a40f"
format_stamp: "Formatted at 2026-08-06 19:43:07 on dist-test-slave-77v9"
server_key: "2e24259e3611971187ceaf6eaa2a3a65"
server_key_iv: "70a29430a27af887c37e4f623b24b8b7"
server_key_version: "encryptionkey@0"
I20260806 19:43:07.548347 31088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "6ce65b879ea64f6799cb460d6234a40f"
format_stamp: "Formatted at 2026-08-06 19:43:07 on dist-test-slave-77v9"
server_key: "2e24259e3611971187ceaf6eaa2a3a65"
server_key_iv: "70a29430a27af887c37e4f623b24b8b7"
server_key_version: "encryptionkey@0"
I20260806 19:43:07.553860 31088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 19:43:07.557636 31103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:07.559036 31088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:43:07.559219 31088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "6ce65b879ea64f6799cb460d6234a40f"
format_stamp: "Formatted at 2026-08-06 19:43:07 on dist-test-slave-77v9"
server_key: "2e24259e3611971187ceaf6eaa2a3a65"
server_key_iv: "70a29430a27af887c37e4f623b24b8b7"
server_key_version: "encryptionkey@0"
I20260806 19:43:07.559444 31088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:07.569473 30946 heartbeater.cc:503] Master 127.21.202.126:42403 was elected leader, sending a full tablet report...
I20260806 19:43:07.579191 31088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:07.583489 31088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:07.583812 31088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:07.584775 31088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:07.586336 31088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:07.586411 31088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:07.586529 31088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:07.586576 31088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:07.622692 31088 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:45983
I20260806 19:43:07.622797 31218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:45983 every 8 connection(s)
I20260806 19:43:07.624028 31088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:43:07.626893 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 31088
I20260806 19:43:07.627126 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:43:07.627641 22313 external_mini_cluster.cc:1428] Setting key 040e0fb41c3bbd3bade485448000104f
Aug 06 19:43:07 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045387, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:43:07.645690 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:07.626788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:44767 (local address 127.21.202.126:42403)
0806 19:43:07.627336 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:07.627343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:07.628659 (+ 1316us) server_negotiation.cc:407] Connection header received
0806 19:43:07.630420 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:07.630425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:07.630530 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:07.630654 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:07.632251 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:07.633133 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:07.634730 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:07.635143 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:07.635287 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:07.639042 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:07.639047 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:07.639051 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:07.639084 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:07.642198 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:07.643161 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:07.643166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:07.643169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:07.643368 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:07.643890 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:07.643894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:07.643897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:07.644180 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:07.644314 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:07.645218 (+ 904us) server_negotiation.cc:299] Negotiation successful
0806 19:43:07.645389 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":258,"threads_started":1}
I20260806 19:43:07.646677 31222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:07.627358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:44767)
0806 19:43:07.628251 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:07.628307 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:07.630138 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:07.630904 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:07.630912 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:07.631501 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:07.632075 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:07.632092 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:07.633305 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:07.633309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:07.634560 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:07.634571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:07.635393 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:07.635400 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:07.635632 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:07.636294 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:07.636335 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:07.638824 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:43:07.642349 (+ 3525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:07.642359 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:07.642377 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:07.643045 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:07.643529 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:07.643534 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:07.643537 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:07.643767 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:07.644386 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:43:07.644395 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:43:07.644782 (+ 387us) client_negotiation.cc:769] Sending connection context
0806 19:43:07.645134 (+ 352us) client_negotiation.cc:240] Negotiation successful
0806 19:43:07.645537 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":366,"threads_started":1}
I20260806 19:43:07.648190 31219 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42403
I20260806 19:43:07.648556 31219 heartbeater.cc:464] Registering TS with master...
I20260806 19:43:07.649451 31219 heartbeater.cc:511] Master 127.21.202.126:42403 requested a full tablet report, sending...
I20260806 19:43:07.651036 30755 ts_manager.cc:194] Registered new tserver with Master: 6ce65b879ea64f6799cb460d6234a40f (127.21.202.67:45983)
I20260806 19:43:07.651924 30755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:44767
I20260806 19:43:07.656199 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab.
Entry for principal oryx/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab.
WARNING: no policy specified for oryx/127.21.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab.
Entry for principal oryx/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab.
WARNING: no policy specified for oryx/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab.
Entry for principal oryx/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab.
I20260806 19:43:07.832168 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 30816
Aug 06 19:43:07 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:43:07.850111 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:07.833570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49954 (local address 127.21.202.65:39763)
0806 19:43:07.834249 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:07.834257 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:07.834285 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:43:07.834397 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:07.834407 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:07.834637 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:07.834793 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:07.836365 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:07.837591 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:07.838990 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:07.839574 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:07.839679 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:07.844602 (+ 4923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:07.844615 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:07.844631 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:07.844667 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:07.847517 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:07.848214 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:07.848221 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:07.848223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:07.848361 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:07.848690 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:07.848693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:07.848695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:07.849115 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:07.849352 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:07.849605 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 19:43:07.849802 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":351,"threads_started":1}
I20260806 19:43:07.851387 30941 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:07.938221 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 30816
I20260806 19:43:07.957692 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 30952
Aug 06 19:43:07 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:43:07.975073 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:07.958864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48886 (local address 127.21.202.66:36733)
0806 19:43:07.959413 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:07.959420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:07.959448 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:43:07.959515 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:07.959521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:07.959734 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:07.959922 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:07.961222 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:07.962556 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:07.963964 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:07.964544 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:07.964727 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:07.969052 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:07.969065 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:07.969087 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:07.969127 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:07.971960 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:07.972613 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:07.972619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:07.972622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:07.972740 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:07.973500 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:07.973503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:07.973505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:07.973898 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:07.974142 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:07.974487 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 19:43:07.974728 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":281,"threads_started":1}
I20260806 19:43:07.976557 31077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:08.059923 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 30952
I20260806 19:43:08.079411 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 31088
Aug 06 19:43:08 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:43:08.098042 31234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:08.080627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56248 (local address 127.21.202.67:45983)
0806 19:43:08.081237 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:08.081244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:08.081276 (+ 32us) server_negotiation.cc:407] Connection header received
0806 19:43:08.081341 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:08.081348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:08.081573 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:08.081720 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:08.083134 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:08.084097 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:08.085736 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:08.086376 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:08.086518 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:08.091285 (+ 4767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:08.091299 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:08.091322 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:08.091374 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:08.094775 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:08.095498 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:08.095507 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:08.095509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:08.095646 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:08.096144 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:08.096149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:08.096151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:08.096546 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:08.096793 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:08.097284 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 19:43:08.097603 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":320,"threads_started":1}
I20260806 19:43:08.099473 31214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:08.187093 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 31088
I20260806 19:43:08.206348 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 30722
I20260806 19:43:08.207703 30785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:08.314090 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 30722
I20260806 19:43:08.338708 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42403
--webserver_interface=127.21.202.126
--webserver_port=36251
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:42403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:43:08.690404 31237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:08.690789 31237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:08.690966 31237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:08.698185 31237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:43:08.698374 31237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:08.698490 31237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:08.698594 31237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:43:08.698660 31237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:43:08.708636 31237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:42403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=36251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:08.710110 31237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:08.711465 31237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:08.723585 31245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:08.723634 31243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:08.723611 31242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:08.724309 31237 server_base.cc:1034] running on GCE node
I20260806 19:43:08.725244 31237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:08.726636 31237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:08.727933 31237 hybrid_clock.cc:648] HybridClock initialized: now 1786045388727880 us; error 49 us; skew 500 ppm
I20260806 19:43:08.728386 31237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:43:08 dist-test-slave-77v9 krb5kdc[30708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045388, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:43:08.732179 31237 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:43:08.733935 31237 webserver.cc:460] Webserver started at http://127.21.202.126:36251/ using document root <none> and password file <none>
I20260806 19:43:08.734730 31237 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:08.734918 31237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:08.740767 31237 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 19:43:08.743888 31252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:08.745745 31237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:43:08.746009 31237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "eef092edec974e4eac904a8ce63c984a"
format_stamp: "Formatted at 2026-08-06 19:43:05 on dist-test-slave-77v9"
server_key: "208dd554d7be4cef5f97bee166aab170"
server_key_iv: "b77fb13f79865264178d965eccf1fcc5"
server_key_version: "encryptionkey@0"
I20260806 19:43:08.747139 31237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:08.772089 31237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:08.776944 31237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:08.777245 31237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:08.799412 31237 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:42403
I20260806 19:43:08.799420 31304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:42403 every 8 connection(s)
I20260806 19:43:08.801088 31237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:43:08.806675 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 31237
I20260806 19:43:08.807708 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:39763
--local_ip_for_outbound_sockets=127.21.202.65
--tserver_master_addrs=127.21.202.126:42403
--webserver_port=34825
--webserver_interface=127.21.202.65
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.202.65
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:43:08.809458 31305 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:43:08.814750 31305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a: Bootstrap starting.
I20260806 19:43:08.835743 31305 log.cc:824] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:43:08.845414 31305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:43:08.845985 31305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a: Bootstrap complete.
I20260806 19:43:08.855588 31305 raft_consensus.cc:348] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef092edec974e4eac904a8ce63c984a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42403 } }
I20260806 19:43:08.856426 31305 raft_consensus.cc:729] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef092edec974e4eac904a8ce63c984a, State: Initialized, Role: FOLLOWER
I20260806 19:43:08.857161 31305 consensus_queue.cc:260] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef092edec974e4eac904a8ce63c984a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42403 } }
I20260806 19:43:08.857429 31305 raft_consensus.cc:388] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:43:08.857551 31305 raft_consensus.cc:482] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:43:08.857749 31305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 1 FOLLOWER]: Advancing to term 2
I20260806 19:43:08.861816 31305 raft_consensus.cc:504] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef092edec974e4eac904a8ce63c984a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42403 } }
I20260806 19:43:08.862347 31305 leader_election.cc:302] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eef092edec974e4eac904a8ce63c984a; no voters:
I20260806 19:43:08.863061 31305 leader_election.cc:288] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 19:43:08.863209 31309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 2 FOLLOWER]: Leader election won for term 2
I20260806 19:43:08.864511 31309 raft_consensus.cc:686] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [term 2 LEADER]: Becoming Leader. State: Replica: eef092edec974e4eac904a8ce63c984a, State: Running, Role: LEADER
I20260806 19:43:08.865142 31309 consensus_queue.cc:237] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef092edec974e4eac904a8ce63c984a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42403 } }
I20260806 19:43:08.865839 31305 sys_catalog.cc:564] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:43:08.868342 31310 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "eef092edec974e4eac904a8ce63c984a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef092edec974e4eac904a8ce63c984a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42403 } } }
I20260806 19:43:08.868855 31310 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [sys.catalog]: This master's current role is: LEADER
I20260806 19:43:08.874128 31311 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [sys.catalog]: SysCatalogTable state changed. Reason: New leader eef092edec974e4eac904a8ce63c984a. Latest consensus state: current_term: 2 leader_uuid: "eef092edec974e4eac904a8ce63c984a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef092edec974e4eac904a8ce63c984a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42403 } } }
I20260806 19:43:08.875392 31311 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a [sys.catalog]: This master's current role is: LEADER
I20260806 19:43:08.877347 31315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:43:08.885442 31315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:43:08.887461 31315 catalog_manager.cc:1259] Loaded cluster ID: 659976977e4f4f86b7461a0ab9a91192
I20260806 19:43:08.887606 31315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:43:08.893360 31326 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 19:43:08.893321 31315 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:43:08.895147 31315 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eef092edec974e4eac904a8ce63c984a: Loaded TSK: 0
I20260806 19:43:08.896380 31315 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 19:43:09.206547 31307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:09.206875 31307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:09.207056 31307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:09.214749 31307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:09.214885 31307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:09.215080 31307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:43:09.225549 31307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.65.keytab
--principal=oryx/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:39763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=34825
--tserver_master_addrs=127.21.202.126:42403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:09.227068 31307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:09.228477 31307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:09.243954 31332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:09.243911 31334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:09.243875 31331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:09.245548 31307 server_base.cc:1034] running on GCE node
I20260806 19:43:09.246255 31307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:09.247473 31307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:09.248713 31307 hybrid_clock.cc:648] HybridClock initialized: now 1786045389248620 us; error 90 us; skew 500 ppm
I20260806 19:43:09.249104 31307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045389, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:43:09.253690 31307 init.cc:377] Logged in from keytab as oryx/127.21.202.65@KRBTEST.COM (short username oryx)
I20260806 19:43:09.255517 31307 webserver.cc:460] Webserver started at http://127.21.202.65:34825/ using document root <none> and password file <none>
I20260806 19:43:09.256577 31307 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:09.256850 31307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:09.264981 31307 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 19:43:09.268362 31341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:09.270151 31307 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:43:09.270390 31307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "c332c665fe094e2c9a4ad57fbafaff4c"
format_stamp: "Formatted at 2026-08-06 19:43:06 on dist-test-slave-77v9"
server_key: "13d6f6adf312a1dd69c494e2fe5c9b7e"
server_key_iv: "9b430ba81abbed7d7235b6e518f1fcca"
server_key_version: "encryptionkey@0"
I20260806 19:43:09.271522 31307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:09.293464 31307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:09.298216 31307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:09.298535 31307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:09.299604 31307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:09.301110 31307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:09.301234 31307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:09.301375 31307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:09.301465 31307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:09.335147 31307 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:39763
I20260806 19:43:09.335366 31454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:39763 every 8 connection(s)
I20260806 19:43:09.336658 31307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:43:09.339643 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 31307
I20260806 19:43:09.340376 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:36733
--local_ip_for_outbound_sockets=127.21.202.66
--tserver_master_addrs=127.21.202.126:42403
--webserver_port=42405
--webserver_interface=127.21.202.66
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.202.66
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:09.355921 31458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:09.340156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:53613)
0806 19:43:09.341104 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:09.341141 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:09.342884 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:09.344243 (+ 1359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:09.344255 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:09.344970 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:09.345504 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:09.345521 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.347121 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.347126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:09.348320 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:09.348342 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.349699 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.349704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:09.349889 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:09.350622 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:09.350662 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:09.354972 (+ 4310us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":408,"threads_started":1}
W20260806 19:43:09.356604 31455 heartbeater.cc:644] Failed to heartbeat to 127.21.202.126:42403 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.202.126:42403: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
W20260806 19:43:09.356626 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:09.339939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:53613 (local address 127.21.202.126:42403)
0806 19:43:09.341002 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:09.341018 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:09.341283 (+ 265us) server_negotiation.cc:407] Connection header received
0806 19:43:09.343407 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:09.343418 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:09.343725 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:09.344090 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:09.345726 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.346941 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.348490 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.349444 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.349639 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:09.355421 (+ 5782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:53613: BlockingRecv error: recv got EOF from 127.21.202.65:53613 (error 108)
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":284,"threads_started":1}
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:09.365022 31458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:09.357212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:34267)
0806 19:43:09.357376 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:09.357403 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:09.357487 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:09.357948 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:09.357953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:09.358298 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:09.358436 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:09.358444 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.359466 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.359471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:09.360376 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:09.360387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.361143 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.361149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:09.361325 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:09.361945 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:09.361962 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:09.364700 (+ 2738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 19:43:09.365172 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:09.357494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:34267 (local address 127.21.202.126:42403)
0806 19:43:09.357682 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:09.357687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:09.357701 (+ 14us) server_negotiation.cc:407] Connection header received
0806 19:43:09.357750 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:09.357754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:09.357808 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:09.357919 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:09.358583 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.359317 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.360528 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.360990 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.361120 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:09.364950 (+ 3830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:34267: BlockingRecv error: recv got EOF from 127.21.202.65:34267 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:09.374024 31458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:09.365808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:60207)
0806 19:43:09.365978 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:09.365998 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:09.366082 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:09.366545 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:09.366550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:09.366850 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:09.366987 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:09.366995 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.368098 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.368104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:09.369087 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:09.369098 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.369809 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.369815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:09.370000 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:09.370586 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:09.370603 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:09.373583 (+ 2980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 19:43:09.374063 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:09.366068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:60207 (local address 127.21.202.126:42403)
0806 19:43:09.366261 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:09.366266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:09.366281 (+ 15us) server_negotiation.cc:407] Connection header received
0806 19:43:09.366328 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:09.366332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:09.366396 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:09.366521 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:09.367169 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.367952 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.369232 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.369640 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.369780 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:09.373824 (+ 4044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:60207: BlockingRecv error: recv got EOF from 127.21.202.65:60207 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 19:43:09.374500 31455 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 19:43:09.697016 31459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:09.697309 31459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:09.697423 31459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:09.704797 31459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:09.705055 31459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:09.705276 31459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:43:09.716317 31459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.66.keytab
--principal=oryx/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:36733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=42405
--tserver_master_addrs=127.21.202.126:42403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:09.717834 31459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:09.719205 31459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:09.733196 31464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:09.733160 31465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:09.733242 31467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:09.734210 31459 server_base.cc:1034] running on GCE node
I20260806 19:43:09.734988 31459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:09.736044 31459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:09.737303 31459 hybrid_clock.cc:648] HybridClock initialized: now 1786045389737239 us; error 52 us; skew 500 ppm
I20260806 19:43:09.737636 31459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045389, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:43:09.740978 31459 init.cc:377] Logged in from keytab as oryx/127.21.202.66@KRBTEST.COM (short username oryx)
I20260806 19:43:09.742400 31459 webserver.cc:460] Webserver started at http://127.21.202.66:42405/ using document root <none> and password file <none>
I20260806 19:43:09.743113 31459 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:09.743240 31459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:09.748906 31459 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:43:09.751583 31474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:09.753240 31459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:43:09.753428 31459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "1d4f42c63c004d6c831b2880f35858dc"
format_stamp: "Formatted at 2026-08-06 19:43:06 on dist-test-slave-77v9"
server_key: "c630dc78dfd2b5af50fb48ee4b63d637"
server_key_iv: "b5b11715ec12f014660fad54aa5d957a"
server_key_version: "encryptionkey@0"
I20260806 19:43:09.754415 31459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:09.771709 31459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:09.776705 31459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:09.777060 31459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:09.778219 31459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:09.779917 31459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:09.780011 31459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:09.780162 31459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:09.780239 31459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:09.817989 31459 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:36733
I20260806 19:43:09.818089 31589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:36733 every 8 connection(s)
I20260806 19:43:09.819484 31459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:43:09.820641 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 31459
I20260806 19:43:09.821349 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:45983
--local_ip_for_outbound_sockets=127.21.202.67
--tserver_master_addrs=127.21.202.126:42403
--webserver_port=39783
--webserver_interface=127.21.202.67
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.202.67
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 19:43:09.837481 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:09.822551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:57865 (local address 127.21.202.126:42403)
0806 19:43:09.822749 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:09.822755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:09.824110 (+ 1355us) server_negotiation.cc:407] Connection header received
0806 19:43:09.826224 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:09.826231 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:09.826332 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:09.826470 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:09.828120 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.828964 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.830862 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.831320 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.831465 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:09.837153 (+ 5688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:57865: BlockingRecv error: recv got EOF from 127.21.202.66:57865 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 19:43:09.838148 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:09.823112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:57865)
0806 19:43:09.823923 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:09.823976 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:09.825923 (+ 1947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:09.826735 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:09.826745 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:09.827304 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:09.827913 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:09.827931 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.829140 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.829152 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:09.830661 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:09.830674 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.831562 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.831569 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:09.831801 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:09.832526 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:09.832561 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:09.836933 (+ 4372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":339,"threads_started":1}
W20260806 19:43:09.838964 31590 heartbeater.cc:644] Failed to heartbeat to 127.21.202.126:42403 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.202.126:42403: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:09.849197 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:09.839570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:44639)
0806 19:43:09.839820 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:09.839843 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:09.839951 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:09.840279 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:09.840287 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:09.840732 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:09.840971 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:09.840985 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.842003 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.842012 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:09.843733 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:09.843750 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.844586 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.844595 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:09.844849 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:09.845472 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:09.845503 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:09.848992 (+ 3489us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":75}
W20260806 19:43:09.849628 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:09.839711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:44639 (local address 127.21.202.126:42403)
0806 19:43:09.839882 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:09.839886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:09.839906 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:43:09.840070 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:09.840074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:09.840132 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:09.840211 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:09.841118 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.841859 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.843891 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.844417 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.844514 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:09.849322 (+ 4808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:44639: BlockingRecv error: recv got EOF from 127.21.202.66:44639 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:09 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:09.859964 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:09.849959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:41819)
0806 19:43:09.850216 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:09.850237 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:09.850337 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:09.850736 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:09.850744 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:09.851193 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:09.851463 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:09.851476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.852708 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.852715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:09.854233 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:09.854248 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.855107 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.855113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:09.855333 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:09.855962 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:09.855984 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:09.859521 (+ 3537us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137}
W20260806 19:43:09.860088 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:09.850135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:41819 (local address 127.21.202.126:42403)
0806 19:43:09.850325 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:09.850331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:09.850349 (+ 18us) server_negotiation.cc:407] Connection header received
0806 19:43:09.850445 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:09.850450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:09.850531 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:09.850669 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:09.851634 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.852557 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.854458 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:09.854962 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:09.855067 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:09.859861 (+ 4794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:41819: BlockingRecv error: recv got EOF from 127.21.202.66:41819 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 19:43:09.860404 31590 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 19:43:10.168434 31593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:10.168782 31593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:10.168994 31593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:10.176447 31593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:10.176604 31593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:10.176777 31593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:43:10.186921 31593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/krb5kdc/oryx_127.21.202.67.keytab
--principal=oryx/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:45983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=39783
--tserver_master_addrs=127.21.202.126:42403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:10.188364 31593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:10.189744 31593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:10.204243 31598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:10.204281 31599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:10.204339 31601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:10.205083 31593 server_base.cc:1034] running on GCE node
I20260806 19:43:10.205789 31593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:10.206852 31593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:10.208091 31593 hybrid_clock.cc:648] HybridClock initialized: now 1786045390208028 us; error 53 us; skew 500 ppm
I20260806 19:43:10.208467 31593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:43:10 dist-test-slave-77v9 krb5kdc[30708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045390, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:43:10.211910 31593 init.cc:377] Logged in from keytab as oryx/127.21.202.67@KRBTEST.COM (short username oryx)
I20260806 19:43:10.213493 31593 webserver.cc:460] Webserver started at http://127.21.202.67:39783/ using document root <none> and password file <none>
I20260806 19:43:10.214176 31593 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:10.214308 31593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:10.220503 31593 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 19:43:10.223820 31608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:10.225576 31593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:43:10.225780 31593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "6ce65b879ea64f6799cb460d6234a40f"
format_stamp: "Formatted at 2026-08-06 19:43:07 on dist-test-slave-77v9"
server_key: "2e24259e3611971187ceaf6eaa2a3a65"
server_key_iv: "70a29430a27af887c37e4f623b24b8b7"
server_key_version: "encryptionkey@0"
I20260806 19:43:10.226843 31593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:10.257642 31593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:10.262097 31593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:10.262404 31593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:10.263481 31593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:10.265028 31593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:10.265211 31593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:10.265365 31593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:10.265461 31593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:10.305327 31593 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:45983
I20260806 19:43:10.305430 31722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:45983 every 8 connection(s)
I20260806 19:43:10.306918 31593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:43:10.314010 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 31593
Aug 06 19:43:10 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:10 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 19:43:10.326331 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:10.309696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:38279 (local address 127.21.202.126:42403)
0806 19:43:10.309912 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:10.309917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:10.311190 (+ 1273us) server_negotiation.cc:407] Connection header received
0806 19:43:10.313351 (+ 2161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:10.313357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:10.313454 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:10.313574 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:10.315596 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.316919 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.318954 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.319485 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.319630 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:10.325863 (+ 6233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:38279: BlockingRecv error: recv got EOF from 127.21.202.67:38279 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 19:43:10.327077 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:10.310157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:38279)
0806 19:43:10.311000 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:10.311045 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:10.313049 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:10.313829 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:10.313839 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:10.314486 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:10.315330 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:10.315365 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.317073 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.317078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:10.318765 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:10.318781 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.319722 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.319728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:10.319940 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:10.320678 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:10.320701 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:10.325606 (+ 4905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":330,"threads_started":1}
W20260806 19:43:10.328207 31723 heartbeater.cc:644] Failed to heartbeat to 127.21.202.126:42403 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.202.126:42403: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
I20260806 19:43:10.333449 31727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:10.315232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40508 (local address 127.21.202.126:42403)
0806 19:43:10.315982 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:10.315987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:10.316003 (+ 16us) server_negotiation.cc:407] Connection header received
0806 19:43:10.316044 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:10.316048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:10.316106 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:10.316200 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:10.318189 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.319321 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.320966 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.321692 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.321841 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:10.325078 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:10.325090 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:10.325114 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:10.325170 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:10.329723 (+ 4553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:10.330529 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:10.330538 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:10.330550 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:10.330790 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:10.331200 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:10.331206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:10.331210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:10.331876 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:10.332176 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:10.332589 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 19:43:10.332997 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":507,"threads_started":1}
Aug 06 19:43:10 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:10 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:10.339165 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:10.328967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:38011)
0806 19:43:10.329198 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:10.329220 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:10.329354 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:10.329688 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:10.329693 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:10.330083 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:10.330258 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:10.330267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.331767 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.331771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:10.333168 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:10.333181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.334121 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.334126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:10.334316 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:10.335882 (+ 1566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:10.335910 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:10.338966 (+ 3056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 19:43:10.339512 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:10.329027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:38011 (local address 127.21.202.126:42403)
0806 19:43:10.329193 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:10.329197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:10.329293 (+ 96us) server_negotiation.cc:407] Connection header received
0806 19:43:10.329485 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:10.329489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:10.329543 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:10.329620 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:10.330412 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.331635 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.333306 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.333963 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.334067 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:10.339247 (+ 5180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:38011: BlockingRecv error: recv got EOF from 127.21.202.67:38011 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 19:43:10 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:10 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:10.348227 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:10.340004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:56783)
0806 19:43:10.340495 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:10.340509 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:10.340633 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:10.341213 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:10.341217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:10.341480 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:10.341623 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:10.341630 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.342678 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.342682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:10.343669 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:10.343681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.344431 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.344436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:10.344577 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:10.345211 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:10.345228 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:10.347990 (+ 2762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379}
W20260806 19:43:10.348591 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:10.340113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:56783 (local address 127.21.202.126:42403)
0806 19:43:10.340381 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:10.340387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:10.340576 (+ 189us) server_negotiation.cc:407] Connection header received
0806 19:43:10.340753 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:10.340758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:10.340903 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:10.340997 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:10.341763 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.342543 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.343811 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.344274 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.344391 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:10.348203 (+ 3812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:56783: BlockingRecv error: recv got EOF from 127.21.202.67:56783 (error 108)
Metrics: {"server-negotiator.queue_time_us":154}
W20260806 19:43:10.348582 31723 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 19:43:10 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:10 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:10.384370 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:10.375202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:56659)
0806 19:43:10.375779 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:10.375796 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:10.375965 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:10.376339 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:10.376343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:10.377009 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:10.377224 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:10.377234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.378433 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.378438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:10.379409 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:10.379420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.380179 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.380185 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:10.380365 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:10.381064 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:10.381082 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:10.384015 (+ 2933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":348,"threads_started":1}
W20260806 19:43:10.384481 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:10.375301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:56659 (local address 127.21.202.126:42403)
0806 19:43:10.375549 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:10.375558 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:10.375966 (+ 408us) server_negotiation.cc:407] Connection header received
0806 19:43:10.376127 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:10.376131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:10.376196 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:10.376301 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:10.377397 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.378279 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.379559 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.379987 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.380121 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:10.384255 (+ 4134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:56659: BlockingRecv error: recv got EOF from 127.21.202.65:56659 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 06 19:43:10 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:10 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:10.874289 31729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:10.861109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:50907)
0806 19:43:10.861648 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:10.861664 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:10.861829 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:10.862207 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:10.862213 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:10.862721 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:10.862924 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:10.862932 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.864087 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.864092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:10.865160 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:10.865172 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.865959 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.865967 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:10.866136 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:10.870988 (+ 4852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:10.871006 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:10.873941 (+ 2935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":286,"threads_started":1}
W20260806 19:43:10.874444 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:10.861185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:50907 (local address 127.21.202.126:42403)
0806 19:43:10.861466 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:10.861472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:10.861753 (+ 281us) server_negotiation.cc:407] Connection header received
0806 19:43:10.861994 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:10.861999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:10.862066 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:10.862172 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:10.863074 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.863938 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.865309 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:10.865754 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:10.865891 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:10.874165 (+ 8274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:50907: BlockingRecv error: recv got EOF from 127.21.202.66:50907 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 06 19:43:11 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:11 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:11.358546 31730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:11.349346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:33187)
0806 19:43:11.350038 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:11.350054 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:11.350227 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:11.350595 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:11.350600 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:11.351146 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:11.351361 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:11.351371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:11.352463 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:11.352469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:11.353505 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:11.353517 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:11.354290 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:11.354297 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:11.354473 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:11.355130 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:11.355148 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:11.358267 (+ 3119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":437,"threads_started":1}
W20260806 19:43:11.358829 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:11.349403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:33187 (local address 127.21.202.126:42403)
0806 19:43:11.349592 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:11.349601 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:11.350150 (+ 549us) server_negotiation.cc:407] Connection header received
0806 19:43:11.350357 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:11.350363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:11.350434 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:11.350543 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:11.351529 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:11.352296 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:11.353653 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:11.354091 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:11.354226 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:11.358530 (+ 4304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:33187: BlockingRecv error: recv got EOF from 127.21.202.67:33187 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 19:43:11 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:11 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:11.394523 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:11.385443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:53945)
0806 19:43:11.385990 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:11.386015 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:11.386184 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:11.386621 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:11.386627 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:11.387137 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:11.387311 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:11.387321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:11.388466 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:11.388471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:11.389459 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:11.389471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:11.390216 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:11.390221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:11.390344 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:11.391001 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:11.391019 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:11.394125 (+ 3106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":452,"threads_started":1}
W20260806 19:43:11.394606 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:11.385642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:53945 (local address 127.21.202.126:42403)
0806 19:43:11.385888 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:11.385895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:11.386121 (+ 226us) server_negotiation.cc:407] Connection header received
0806 19:43:11.386332 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:11.386338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:11.386429 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:11.386551 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:11.387490 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:11.388331 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:11.389608 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:11.390065 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:11.390162 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:11.394335 (+ 4173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:53945: BlockingRecv error: recv got EOF from 127.21.202.65:53945 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 19:43:11 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:11 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:11.885618 31732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:11.875485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:41549)
0806 19:43:11.876027 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:11.876048 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:11.876237 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:11.876582 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:11.876587 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:11.877224 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:11.877425 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:11.877435 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:11.878686 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:11.878690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:11.879634 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:11.879645 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:11.880474 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:11.880479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:11.880613 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:11.881399 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:11.881420 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:11.885199 (+ 3779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":292,"threads_started":1}
W20260806 19:43:11.885758 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:11.875580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:41549 (local address 127.21.202.126:42403)
0806 19:43:11.875855 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:11.875862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:11.876222 (+ 360us) server_negotiation.cc:407] Connection header received
0806 19:43:11.876332 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:11.876336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:11.876411 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:11.876498 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:11.877571 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:11.878534 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:11.879787 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:11.880276 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:11.880402 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:11.885471 (+ 5069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:41549: BlockingRecv error: recv got EOF from 127.21.202.66:41549 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 06 19:43:12 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:12 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:12.368271 31733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:12.359718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:53269)
0806 19:43:12.360316 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:12.360334 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:12.360458 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:12.360837 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:12.360841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:12.361291 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:12.361480 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:12.361490 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:12.362569 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:12.362573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:12.363477 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:12.363488 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:12.364234 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:12.364238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:12.364347 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:12.364993 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:12.365010 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:12.368017 (+ 3007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":333,"threads_started":1}
W20260806 19:43:12.368525 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:12.359836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:53269 (local address 127.21.202.126:42403)
0806 19:43:12.360016 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:12.360020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:12.360406 (+ 386us) server_negotiation.cc:407] Connection header received
0806 19:43:12.360601 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:12.360606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:12.360674 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:12.360790 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:12.361635 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:12.362423 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:12.363625 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:12.364100 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:12.364194 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:12.368223 (+ 4029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:53269: BlockingRecv error: recv got EOF from 127.21.202.67:53269 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 19:43:12 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:12 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:12.404604 31735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:12.395613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:43377)
0806 19:43:12.396258 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:12.396278 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:12.396436 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:12.396865 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:12.396873 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:12.397449 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:12.397661 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:12.397672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:12.398796 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:12.398801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:12.399748 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:12.399761 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:12.400452 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:12.400456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:12.400593 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:12.401284 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:12.401303 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:12.404327 (+ 3024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":340,"threads_started":1}
W20260806 19:43:12.404968 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:12.395708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:43377 (local address 127.21.202.126:42403)
0806 19:43:12.395948 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:12.395958 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:12.396421 (+ 463us) server_negotiation.cc:407] Connection header received
0806 19:43:12.396582 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:12.396588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:12.396685 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:12.396788 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:12.397830 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:12.398604 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:12.399911 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:12.400304 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:12.400409 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:12.404572 (+ 4163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:43377: BlockingRecv error: recv got EOF from 127.21.202.65:43377 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 19:43:12 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:12 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:12.896641 31736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:12.887028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:37901)
0806 19:43:12.887745 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:12.887765 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:12.887952 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:12.888425 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:12.888431 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:12.888981 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:12.889232 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:12.889247 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:12.890417 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:12.890421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:12.891352 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:12.891364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:12.892102 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:12.892107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:12.892234 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:12.892972 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:12.892990 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:12.896376 (+ 3386us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":333,"threads_started":1}
W20260806 19:43:12.896860 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:12.887155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:37901 (local address 127.21.202.126:42403)
0806 19:43:12.887393 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:12.887401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:12.887868 (+ 467us) server_negotiation.cc:407] Connection header received
0806 19:43:12.888169 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:12.888175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:12.888268 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:12.888383 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:12.889427 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:12.890257 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:12.891491 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:12.891945 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:12.892041 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:12.896589 (+ 4548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:37901: BlockingRecv error: recv got EOF from 127.21.202.66:37901 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 19:43:13 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:13 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:13.378489 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:13.369508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:34621)
0806 19:43:13.370136 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:13.370153 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:13.370275 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:13.370595 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:13.370598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:13.370977 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:13.371147 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:13.371157 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:13.372251 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:13.372255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:13.373262 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:13.373275 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:13.374075 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:13.374080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:13.374214 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:13.374924 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:13.374942 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:13.378184 (+ 3242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":365,"threads_started":1}
W20260806 19:43:13.378662 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:13.369581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:34621 (local address 127.21.202.126:42403)
0806 19:43:13.369773 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:13.369778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:13.370263 (+ 485us) server_negotiation.cc:407] Connection header received
0806 19:43:13.370388 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:13.370392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:13.370461 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:13.370547 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:13.371322 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:13.372103 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:13.373418 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:13.373914 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:13.374029 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:13.378410 (+ 4381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:34621: BlockingRecv error: recv got EOF from 127.21.202.67:34621 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 19:43:13 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:13 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:13.414925 31738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:13.405773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:57161)
0806 19:43:13.406289 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:13.406306 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:13.406433 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:13.406746 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:13.406749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:13.407124 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:13.407293 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:13.407302 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:13.408326 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:13.408331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:13.409560 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:13.409574 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:13.410384 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:13.410390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:13.410528 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:13.411174 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:13.411193 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:13.414679 (+ 3486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":290,"threads_started":1}
W20260806 19:43:13.415160 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:13.405844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:57161 (local address 127.21.202.126:42403)
0806 19:43:13.406126 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:13.406134 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:13.406403 (+ 269us) server_negotiation.cc:407] Connection header received
0806 19:43:13.406548 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:13.406552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:13.406617 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:13.406700 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:13.407436 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:13.408176 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:13.409754 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:13.410217 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:13.410339 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:13.414907 (+ 4568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:57161: BlockingRecv error: recv got EOF from 127.21.202.65:57161 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 06 19:43:13 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:13 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:13.907203 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:13.897918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:35353)
0806 19:43:13.898424 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:13.898441 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:13.898573 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:13.898921 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:13.898926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:13.899466 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:13.899676 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:13.899691 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:13.901189 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:13.901199 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:13.902297 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:13.902310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:13.903060 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:13.903065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:13.903231 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:13.903847 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:13.903866 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:13.906956 (+ 3090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":280,"threads_started":1}
W20260806 19:43:13.907466 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:13.897995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:35353 (local address 127.21.202.126:42403)
0806 19:43:13.898249 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:13.898255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:13.898530 (+ 275us) server_negotiation.cc:407] Connection header received
0806 19:43:13.898678 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:13.898682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:13.898746 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:13.898831 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:13.899923 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:13.901025 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:13.902443 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:13.902899 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:13.902997 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:13.907191 (+ 4194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:35353: BlockingRecv error: recv got EOF from 127.21.202.66:35353 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 19:43:14 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:14 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:14.388528 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:14.379537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:43129)
0806 19:43:14.380053 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:14.380069 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:14.380232 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:14.380587 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:14.380591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:14.381039 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:14.381194 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:14.381202 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:14.382262 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:14.382270 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:14.383247 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:14.383258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:14.383997 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:14.384003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:14.384184 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:14.384981 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:14.385001 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:14.388185 (+ 3184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":282,"threads_started":1}
W20260806 19:43:14.388711 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:14.379635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:43129 (local address 127.21.202.126:42403)
0806 19:43:14.379848 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:14.379857 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:14.380163 (+ 306us) server_negotiation.cc:407] Connection header received
0806 19:43:14.380361 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:14.380366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:14.380442 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:14.380547 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:14.381349 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:14.382128 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:14.383405 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:14.383835 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:14.383973 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:14.388429 (+ 4456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:43129: BlockingRecv error: recv got EOF from 127.21.202.67:43129 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 19:43:14 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:14 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:14.424312 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:14.416147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:52677)
0806 19:43:14.416749 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:14.416766 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:14.416958 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:14.417258 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:14.417261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:14.417689 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:14.417880 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:14.417890 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:14.418863 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:14.418866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:14.419792 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:14.419804 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:14.420466 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:14.420470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:14.420594 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:14.421260 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:14.421278 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:14.424058 (+ 2780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":323,"threads_started":1}
W20260806 19:43:14.424717 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:14.416183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:52677 (local address 127.21.202.126:42403)
0806 19:43:14.416374 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:14.416384 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:14.416880 (+ 496us) server_negotiation.cc:407] Connection header received
0806 19:43:14.417066 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:14.417070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:14.417134 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:14.417216 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:14.418010 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:14.418741 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:14.419919 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:14.420335 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:14.420433 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:14.424414 (+ 3981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:52677: BlockingRecv error: recv got EOF from 127.21.202.65:52677 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 19:43:14 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:14 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:14.917649 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:14.908407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:45957)
0806 19:43:14.909047 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:14.909064 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:14.909220 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:14.909585 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:14.909591 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:14.910041 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:14.910201 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:14.910216 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:14.911274 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:14.911282 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:14.912219 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:14.912231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:14.913036 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:14.913042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:14.913222 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:14.913871 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:14.913889 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:14.917375 (+ 3486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":294,"threads_started":1}
W20260806 19:43:14.918048 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:14.908615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:45957 (local address 127.21.202.126:42403)
0806 19:43:14.908845 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:14.908852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:14.909157 (+ 305us) server_negotiation.cc:407] Connection header received
0806 19:43:14.909366 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:14.909372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:14.909449 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:14.909550 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:14.910356 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:14.911130 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:14.912370 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:14.912888 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:14.913024 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:14.917674 (+ 4650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:45957: BlockingRecv error: recv got EOF from 127.21.202.66:45957 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 19:43:15 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:15 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:15.399636 31743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:15.389637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:39285)
0806 19:43:15.390398 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:15.390415 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:15.390589 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:15.390949 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:15.390955 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:15.391380 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:15.391547 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:15.391556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:15.392591 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:15.392598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:15.393575 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:15.393587 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:15.394347 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:15.394353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:15.394542 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:15.395241 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:15.395259 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:15.399323 (+ 4064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":430,"threads_started":1}
W20260806 19:43:15.399817 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:15.389851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:39285 (local address 127.21.202.126:42403)
0806 19:43:15.390224 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:15.390230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:15.390522 (+ 292us) server_negotiation.cc:407] Connection header received
0806 19:43:15.390723 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:15.390733 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:15.390796 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:15.390907 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:15.391706 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:15.392438 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:15.393743 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:15.394184 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:15.394319 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:15.399557 (+ 5238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:39285: BlockingRecv error: recv got EOF from 127.21.202.67:39285 (error 108)
Metrics: {"server-negotiator.queue_time_us":186}
Aug 06 19:43:15 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:15 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:15.437868 31744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:15.425478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:56093)
0806 19:43:15.426057 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:15.426074 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:15.426196 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:15.426700 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:15.426705 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:15.427353 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:15.427602 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:15.427618 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:15.429353 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:15.429361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:15.431029 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:15.431049 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:15.432189 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:15.432197 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:15.432411 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:15.433235 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:15.433259 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:15.437635 (+ 4376us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":329,"threads_started":1}
W20260806 19:43:15.438196 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:15.425533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:56093 (local address 127.21.202.126:42403)
0806 19:43:15.425796 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:15.425805 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:15.426189 (+ 384us) server_negotiation.cc:407] Connection header received
0806 19:43:15.426306 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:15.426310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:15.426505 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:15.426600 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:15.427776 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:15.429155 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:15.431203 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:15.431987 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:15.432141 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:15.437862 (+ 5721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:56093: BlockingRecv error: recv got EOF from 127.21.202.65:56093 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 19:43:15 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:15 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:15.932039 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:15.918678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:56213)
0806 19:43:15.919181 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:15.919198 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:15.919317 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:15.919608 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:15.919612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:15.920013 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:15.920178 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:15.920186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:15.921297 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:15.921302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:15.922351 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:15.922366 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:15.923123 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:15.923128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:15.923277 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:15.928556 (+ 5279us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:15.928578 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:15.931809 (+ 3231us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":285,"threads_started":1}
W20260806 19:43:15.932467 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:15.918752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:56213 (local address 127.21.202.126:42403)
0806 19:43:15.919009 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:15.919016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:15.919280 (+ 264us) server_negotiation.cc:407] Connection header received
0806 19:43:15.919415 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:15.919418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:15.919483 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:15.919565 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:15.920298 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:15.921098 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:15.922509 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:15.922941 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:15.923038 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:15.932183 (+ 9145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:56213: BlockingRecv error: recv got EOF from 127.21.202.66:56213 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 19:43:16 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:16 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:16.410297 31746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:16.400795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:47511)
0806 19:43:16.401418 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:16.401435 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:16.401606 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:16.401979 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:16.401984 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:16.402420 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:16.402591 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:16.402599 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:16.403674 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:16.403680 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:16.404594 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:16.404606 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:16.405520 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:16.405525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:16.405769 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:16.406533 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:16.406559 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:16.409965 (+ 3406us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":327,"threads_started":1}
W20260806 19:43:16.410426 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:16.400957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:47511 (local address 127.21.202.126:42403)
0806 19:43:16.401204 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:16.401213 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:16.401538 (+ 325us) server_negotiation.cc:407] Connection header received
0806 19:43:16.401741 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:16.401746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:16.401815 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:16.401948 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:16.402747 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:16.403524 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:16.404749 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:16.405307 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:16.405488 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:16.410167 (+ 4679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:47511: BlockingRecv error: recv got EOF from 127.21.202.67:47511 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 19:43:16 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:16 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:16.448773 31747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:16.438924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:46795)
0806 19:43:16.439664 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:16.439681 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:16.439865 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:16.440258 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:16.440263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:16.440725 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:16.440920 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:16.440937 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:16.442030 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:16.442034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:16.443642 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:16.443660 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:16.444419 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:16.444425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:16.444551 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:16.445276 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:16.445297 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:16.448423 (+ 3126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":345,"threads_started":1}
W20260806 19:43:16.448982 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:16.439124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:46795 (local address 127.21.202.126:42403)
0806 19:43:16.439487 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:16.439495 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:16.439789 (+ 294us) server_negotiation.cc:407] Connection header received
0806 19:43:16.440021 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:16.440026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:16.440105 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:16.440222 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:16.441088 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:16.441839 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:16.443796 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:16.444239 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:16.444334 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:16.448661 (+ 4327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:46795: BlockingRecv error: recv got EOF from 127.21.202.65:46795 (error 108)
Metrics: {"server-negotiator.queue_time_us":160}
Aug 06 19:43:16 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:16 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:16.943219 31748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:16.933197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:34097)
0806 19:43:16.933936 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:16.933962 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:16.934132 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:16.934592 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:16.934597 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:16.935202 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:16.935395 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:16.935404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:16.936492 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:16.936497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:16.937523 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:16.937543 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:16.938437 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:16.938443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:16.938620 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:16.939294 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:16.939319 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:16.942799 (+ 3480us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":311,"threads_started":1}
W20260806 19:43:16.943403 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:16.933809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:34097 (local address 127.21.202.126:42403)
0806 19:43:16.933990 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:16.933995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:16.934043 (+ 48us) server_negotiation.cc:407] Connection header received
0806 19:43:16.934287 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:16.934293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:16.934418 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:16.934558 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:16.935544 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:16.936337 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:16.937675 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:16.938234 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:16.938407 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:16.943090 (+ 4683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:34097: BlockingRecv error: recv got EOF from 127.21.202.66:34097 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 19:43:17 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:17 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:17.420919 31749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:17.411353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:58963)
0806 19:43:17.411995 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:17.412012 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:17.412186 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:17.412565 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:17.412571 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:17.413203 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:17.413396 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:17.413408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:17.414535 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:17.414545 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:17.415551 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:17.415563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:17.416402 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:17.416409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:17.416595 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:17.417218 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:17.417237 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:17.420438 (+ 3201us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":285,"threads_started":1}
W20260806 19:43:17.420919 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:17.411428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:58963 (local address 127.21.202.126:42403)
0806 19:43:17.411673 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:17.411682 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:17.412112 (+ 430us) server_negotiation.cc:407] Connection header received
0806 19:43:17.412321 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:17.412327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:17.412414 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:17.412530 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:17.413581 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:17.414381 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:17.415710 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:17.416226 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:17.416392 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:17.420660 (+ 4268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:58963: BlockingRecv error: recv got EOF from 127.21.202.67:58963 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 19:43:17 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:17 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:17.460043 31750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:17.449871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:59649)
0806 19:43:17.450491 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:17.450509 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:17.450629 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:17.450983 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:17.450988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:17.451423 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:17.451582 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:17.451590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:17.452880 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:17.452889 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:17.453909 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:17.453922 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:17.454792 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:17.454797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:17.454933 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:17.455636 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:17.455664 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:17.459695 (+ 4031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":316,"threads_started":1}
W20260806 19:43:17.460219 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:17.450225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:59649 (local address 127.21.202.126:42403)
0806 19:43:17.450437 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:17.450443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:17.450616 (+ 173us) server_negotiation.cc:407] Connection header received
0806 19:43:17.450725 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:17.450729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:17.450834 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:17.450925 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:17.451718 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:17.452610 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:17.454066 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:17.454582 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:17.454692 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:17.459926 (+ 5234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:59649: BlockingRecv error: recv got EOF from 127.21.202.65:59649 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 19:43:17 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:17 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:17.954404 31751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:17.944418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:55687)
0806 19:43:17.945336 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:17.945356 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:17.945490 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:17.945906 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:17.945910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:17.946343 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:17.946517 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:17.946526 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:17.947891 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:17.947896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:17.949074 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:17.949088 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:17.949866 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:17.949871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:17.950056 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:17.950815 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:17.950835 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:17.954161 (+ 3326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":570,"threads_started":1}
W20260806 19:43:17.954608 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:17.944470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:55687 (local address 127.21.202.126:42403)
0806 19:43:17.944683 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:17.944693 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:17.945468 (+ 775us) server_negotiation.cc:407] Connection header received
0806 19:43:17.945640 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:17.945645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:17.945713 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:17.945808 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:17.946692 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:17.947692 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:17.949218 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:17.949662 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:17.949767 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:17.954369 (+ 4602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:55687: BlockingRecv error: recv got EOF from 127.21.202.66:55687 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 19:43:18 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:18 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:18.431485 31752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:18.422203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:55547)
0806 19:43:18.422810 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:18.422836 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:18.422996 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:18.423431 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:18.423435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:18.423981 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:18.424219 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:18.424237 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:18.425368 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:18.425372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:18.426300 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:18.426311 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:18.427299 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:18.427305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:18.427442 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:18.428105 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:18.428125 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:18.431191 (+ 3066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":282,"threads_started":1}
W20260806 19:43:18.431674 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:18.422274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:55547 (local address 127.21.202.126:42403)
0806 19:43:18.422552 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:18.422561 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:18.422985 (+ 424us) server_negotiation.cc:407] Connection header received
0806 19:43:18.423171 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:18.423178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:18.423265 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:18.423379 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:18.424443 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:18.425205 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:18.426525 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:18.427074 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:18.427178 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:18.431414 (+ 4236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:55547: BlockingRecv error: recv got EOF from 127.21.202.67:55547 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 19:43:18 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:18 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:18.470644 31753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:18.461288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:48911)
0806 19:43:18.461863 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:18.461880 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:18.461998 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:18.462362 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:18.462365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:18.462762 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:18.462952 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:18.462961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:18.464036 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:18.464042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:18.465224 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:18.465240 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:18.466119 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:18.466125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:18.466293 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:18.466985 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:18.467005 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:18.470391 (+ 3386us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":322,"threads_started":1}
W20260806 19:43:18.470984 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:18.461422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:48911 (local address 127.21.202.126:42403)
0806 19:43:18.461627 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:18.461631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:18.461990 (+ 359us) server_negotiation.cc:407] Connection header received
0806 19:43:18.462156 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:18.462160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:18.462225 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:18.462317 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:18.463091 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:18.463840 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:18.465387 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:18.465891 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:18.466046 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:18.470636 (+ 4590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:48911: BlockingRecv error: recv got EOF from 127.21.202.65:48911 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 19:43:18 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:18 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:18.965039 31754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:18.955592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:55863)
0806 19:43:18.956185 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:18.956201 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:18.956372 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:18.956748 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:18.956754 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:18.957217 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:18.957394 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:18.957403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:18.958508 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:18.958515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:18.959539 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:18.959551 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:18.960305 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:18.960310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:18.960503 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:18.961259 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:18.961280 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:18.964641 (+ 3361us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":364,"threads_started":1}
W20260806 19:43:18.965167 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:18.955618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:55863 (local address 127.21.202.126:42403)
0806 19:43:18.955833 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:18.955841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:18.956298 (+ 457us) server_negotiation.cc:407] Connection header received
0806 19:43:18.956506 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:18.956511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:18.956608 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:18.956716 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:18.957549 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:18.958370 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:18.959689 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:18.960146 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:18.960290 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:18.964891 (+ 4601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:55863: BlockingRecv error: recv got EOF from 127.21.202.66:55863 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 19:43:19 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:19 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:19.443272 31755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:19.433358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:50565)
0806 19:43:19.433922 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:19.433942 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:19.434108 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:19.434665 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:19.434673 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:19.435212 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:19.435454 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:19.435470 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:19.437038 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:19.437043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:19.438440 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:19.438452 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:19.439188 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:19.439194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:19.439369 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:19.439918 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:19.439936 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:19.442991 (+ 3055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":305,"threads_started":1}
W20260806 19:43:19.443382 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:19.433500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:50565 (local address 127.21.202.126:42403)
0806 19:43:19.434053 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:19.434062 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:19.434086 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:43:19.434267 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:19.434272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:19.434359 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:19.434425 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:19.435650 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:19.436901 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:19.438582 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:19.439026 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:19.439168 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:19.443195 (+ 4027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:50565: BlockingRecv error: recv got EOF from 127.21.202.67:50565 (error 108)
Metrics: {"server-negotiator.queue_time_us":403}
Aug 06 19:43:19 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:19 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:19.481726 31756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:19.471914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:38419)
0806 19:43:19.472509 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:19.472526 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:19.472639 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:19.473035 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:19.473038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:19.473403 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:19.473546 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:19.473553 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:19.474801 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:19.474806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:19.475979 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:19.475993 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:19.476891 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:19.476898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:19.477094 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:19.477816 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:19.477840 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:19.481347 (+ 3507us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":309,"threads_started":1}
W20260806 19:43:19.481914 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:19.472101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:38419 (local address 127.21.202.126:42403)
0806 19:43:19.472343 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:19.472348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:19.472638 (+ 290us) server_negotiation.cc:407] Connection header received
0806 19:43:19.472749 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:19.472753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:19.472901 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:19.472996 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:19.473712 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:19.474618 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:19.476124 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:19.476637 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:19.476861 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:19.481577 (+ 4716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:38419: BlockingRecv error: recv got EOF from 127.21.202.65:38419 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 19:43:19 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:19 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:19.975448 31757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:19.966154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:57485)
0806 19:43:19.966748 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:19.966765 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:19.966984 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:19.967397 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:19.967402 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:19.967814 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:19.967972 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:19.967980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:19.969315 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:19.969322 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:19.970532 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:19.970545 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:19.971280 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:19.971285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:19.971434 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:19.972074 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:19.972092 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:19.975112 (+ 3020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":350,"threads_started":1}
W20260806 19:43:19.975569 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:19.966227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:57485 (local address 127.21.202.126:42403)
0806 19:43:19.966576 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:19.966583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:19.966887 (+ 304us) server_negotiation.cc:407] Connection header received
0806 19:43:19.967155 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:19.967163 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:19.967253 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:19.967372 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:19.968129 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:19.969124 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:19.970657 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:19.971105 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:19.971204 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:19.975329 (+ 4125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:57485: BlockingRecv error: recv got EOF from 127.21.202.66:57485 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 06 19:43:20 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:20 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:20.454409 31758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:20.444387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:60117)
0806 19:43:20.445431 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:20.445448 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:20.445619 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:20.446001 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:20.446005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:20.446444 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:20.446606 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:20.446615 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:20.447676 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:20.447682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:20.448670 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:20.448681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:20.449473 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:20.449479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:20.449662 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:20.450394 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:20.450435 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:20.454106 (+ 3671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":702,"threads_started":1}
W20260806 19:43:20.455015 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:20.444643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:60117 (local address 127.21.202.126:42403)
0806 19:43:20.445278 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:20.445285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:20.445546 (+ 261us) server_negotiation.cc:407] Connection header received
0806 19:43:20.445756 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:20.445761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:20.445833 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:20.445975 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:20.446766 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:20.447522 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:20.448843 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:20.449301 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:20.449446 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:20.454696 (+ 5250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:60117: BlockingRecv error: recv got EOF from 127.21.202.67:60117 (error 108)
Metrics: {"server-negotiator.queue_time_us":249}
Aug 06 19:43:20 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:20 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:20.492925 31759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:20.482907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:54209)
0806 19:43:20.483553 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:20.483576 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:20.483773 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:20.484210 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:20.484229 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:20.484738 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:20.485021 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:20.485032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:20.486206 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:20.486215 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:20.487268 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:20.487282 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:20.488113 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:20.488120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:20.488338 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:20.489166 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:20.489187 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:20.492533 (+ 3346us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":341,"threads_started":1}
W20260806 19:43:20.493098 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:20.483270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:54209 (local address 127.21.202.126:42403)
0806 19:43:20.483545 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:20.483552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:20.483679 (+ 127us) server_negotiation.cc:407] Connection header received
0806 19:43:20.483953 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:20.483957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:20.484045 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:20.484164 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:20.485191 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:20.486048 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:20.487439 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:20.487935 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:20.488085 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:20.492772 (+ 4687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:54209: BlockingRecv error: recv got EOF from 127.21.202.65:54209 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 06 19:43:20 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:20 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:20.990571 31760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:20.976453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:41847)
0806 19:43:20.977366 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:20.977383 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:20.977535 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:20.977931 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:20.977936 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:20.978484 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:20.978682 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:20.978691 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:20.979732 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:20.979738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:20.980635 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:20.980646 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:20.981452 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:20.981458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:20.981642 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:20.987065 (+ 5423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:20.987083 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:20.990267 (+ 3184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":790,"thread_start_us":648,"threads_started":1}
W20260806 19:43:20.990975 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:20.976579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:41847 (local address 127.21.202.126:42403)
0806 19:43:20.976865 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:20.976877 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:20.977467 (+ 590us) server_negotiation.cc:407] Connection header received
0806 19:43:20.977665 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:20.977670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:20.977761 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:20.977896 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:20.978828 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:20.979581 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:20.980779 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:20.981284 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:20.981435 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:20.990655 (+ 9220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:41847: BlockingRecv error: recv got EOF from 127.21.202.66:41847 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 19:43:21 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:21 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:21.464176 31761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:21.455387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:42841)
0806 19:43:21.455927 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:21.455943 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:21.456108 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:21.456446 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:21.456453 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:21.456968 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:21.457152 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:21.457160 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:21.458242 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:21.458249 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:21.459140 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:21.459151 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:21.459888 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:21.459895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:21.460083 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:21.460798 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:21.460835 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:21.463918 (+ 3083us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":278,"threads_started":1}
W20260806 19:43:21.464977 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:21.455475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:42841 (local address 127.21.202.126:42403)
0806 19:43:21.455726 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:21.455736 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:21.456030 (+ 294us) server_negotiation.cc:407] Connection header received
0806 19:43:21.456230 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:21.456235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:21.456302 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:21.456402 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:21.457297 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:21.458103 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:21.459280 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:21.459719 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:21.459864 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:21.464227 (+ 4363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:42841: BlockingRecv error: recv got EOF from 127.21.202.67:42841 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 06 19:43:21 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:21 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:21.503564 31762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:21.493966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:43153)
0806 19:43:21.494480 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:21.494497 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:21.494622 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:21.495000 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:21.495004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:21.495448 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:21.495609 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:21.495617 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:21.497094 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:21.497101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:21.498502 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:21.498517 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:21.499371 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:21.499379 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:21.499587 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:21.500262 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:21.500283 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:21.503313 (+ 3030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":304,"threads_started":1}
W20260806 19:43:21.503827 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:21.494085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:43153 (local address 127.21.202.126:42403)
0806 19:43:21.494363 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:21.494370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:21.494605 (+ 235us) server_negotiation.cc:407] Connection header received
0806 19:43:21.494727 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:21.494731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:21.494817 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:21.494906 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:21.495788 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:21.496933 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:21.498661 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:21.499184 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:21.499333 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:21.503523 (+ 4190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:43153: BlockingRecv error: recv got EOF from 127.21.202.65:43153 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 06 19:43:21 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:21 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:22.001340 31763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:21.991667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:57347)
0806 19:43:21.992205 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:21.992222 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:21.992343 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:21.992637 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:21.992641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:21.993111 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:21.993273 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:21.993285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:21.994451 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:21.994456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:21.995511 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:21.995525 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:21.996315 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:21.996320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:21.996502 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:21.997304 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:21.997327 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:22.001006 (+ 3679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":334,"threads_started":1}
W20260806 19:43:22.001482 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:21.991744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:57347 (local address 127.21.202.126:42403)
0806 19:43:21.992053 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:21.992061 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:21.992332 (+ 271us) server_negotiation.cc:407] Connection header received
0806 19:43:21.992445 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:21.992448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:21.992512 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:21.992597 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:21.993443 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:21.994262 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:21.995671 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:21.996144 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:21.996259 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:22.001258 (+ 4999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:57347: BlockingRecv error: recv got EOF from 127.21.202.66:57347 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 06 19:43:22 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:22 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:22.474108 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:22.465198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:44037)
0806 19:43:22.465784 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:22.465800 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:22.465916 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:22.466287 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:22.466291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:22.466723 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:22.466890 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:22.466898 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:22.468122 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:22.468126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:22.469129 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:22.469141 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:22.469957 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:22.469961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:22.470095 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:22.470676 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:22.470703 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:22.473762 (+ 3059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":358,"threads_started":1}
W20260806 19:43:22.474278 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:22.465268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:44037 (local address 127.21.202.126:42403)
0806 19:43:22.465495 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:22.465500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:22.465883 (+ 383us) server_negotiation.cc:407] Connection header received
0806 19:43:22.466004 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:22.466008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:22.466073 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:22.466165 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:22.467081 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:22.467975 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:22.469281 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:22.469815 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:22.469921 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:22.474003 (+ 4082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:44037: BlockingRecv error: recv got EOF from 127.21.202.67:44037 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 06 19:43:22 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:22 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:22.513862 31765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:22.504482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:35289)
0806 19:43:22.505181 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:22.505200 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:22.505375 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:22.505785 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:22.505790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:22.506225 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:22.506398 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:22.506407 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:22.507564 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:22.507571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:22.508608 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:22.508620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:22.509551 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:22.509558 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:22.509760 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:22.510466 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:22.510485 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:22.513450 (+ 2965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":380,"threads_started":1}
W20260806 19:43:22.513952 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:22.504650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:35289 (local address 127.21.202.126:42403)
0806 19:43:22.504936 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:22.504945 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:22.505307 (+ 362us) server_negotiation.cc:407] Connection header received
0806 19:43:22.505514 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:22.505520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:22.505624 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:22.505739 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:22.506574 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:22.507409 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:22.508767 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:22.509332 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:22.509480 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:22.513672 (+ 4192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:35289: BlockingRecv error: recv got EOF from 127.21.202.65:35289 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 06 19:43:23 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:23 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:23.011234 31766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:23.002447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:51605)
0806 19:43:23.003100 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:23.003122 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:23.003296 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:23.003656 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:23.003661 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:23.004083 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:23.004247 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:23.004256 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:23.005362 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:23.005368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:23.006302 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:23.006312 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:23.007091 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:23.007098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:23.007268 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:23.007872 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:23.007891 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:23.010972 (+ 3081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":364,"threads_started":1}
W20260806 19:43:23.011415 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:23.002657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:51605 (local address 127.21.202.126:42403)
0806 19:43:23.002923 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:23.002932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:23.003219 (+ 287us) server_negotiation.cc:407] Connection header received
0806 19:43:23.003433 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:23.003439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:23.003515 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:23.003620 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:23.004398 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:23.005212 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:23.006469 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:23.006909 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:23.007050 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:23.011162 (+ 4112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:51605: BlockingRecv error: recv got EOF from 127.21.202.66:51605 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 06 19:43:23 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:23 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:23.484524 31767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:23.475210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:52897)
0806 19:43:23.475913 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:23.475931 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:23.476052 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:23.476354 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:23.476357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:23.476944 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:23.477189 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:23.477200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:23.478342 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:23.478347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:23.479369 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:23.479382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:23.480229 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:23.480238 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:23.480428 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:23.481147 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:23.481166 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:23.484137 (+ 2971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":399,"threads_started":1}
W20260806 19:43:23.484660 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:23.475295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:52897 (local address 127.21.202.126:42403)
0806 19:43:23.475549 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:23.475556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:23.476010 (+ 454us) server_negotiation.cc:407] Connection header received
0806 19:43:23.476154 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:23.476158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:23.476222 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:23.476305 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:23.477343 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:23.478051 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:23.479556 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:23.480032 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:23.480180 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:23.484357 (+ 4177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:52897: BlockingRecv error: recv got EOF from 127.21.202.67:52897 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 06 19:43:23 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:23 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:23.523511 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:23.515034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:48519)
0806 19:43:23.515621 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:23.515639 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:23.515753 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:23.516196 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:23.516201 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:23.516679 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:23.516950 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:23.516963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:23.518101 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:23.518105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:23.519000 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:23.519011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:23.519713 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:23.519717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:23.519831 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:23.520448 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:23.520465 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:23.523245 (+ 2780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":334,"threads_started":1}
W20260806 19:43:23.523821 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:23.515188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:48519 (local address 127.21.202.126:42403)
0806 19:43:23.515507 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:23.515513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:23.515725 (+ 212us) server_negotiation.cc:407] Connection header received
0806 19:43:23.515906 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:23.515910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:23.516019 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:23.516126 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:23.517107 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:23.517963 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:23.519150 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:23.519585 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:23.519686 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:23.523435 (+ 3749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:48519: BlockingRecv error: recv got EOF from 127.21.202.65:48519 (error 108)
Metrics: {"server-negotiator.queue_time_us":151}
Aug 06 19:43:24 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:24 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:24.022181 31769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:24.012467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:37109)
0806 19:43:24.013001 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:24.013018 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:24.013139 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:24.013492 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:24.013496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:24.013913 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:24.014176 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:24.014186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:24.015218 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:24.015224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:24.016318 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:24.016331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:24.017215 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:24.017219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:24.017363 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:24.018242 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:24.018268 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:24.021924 (+ 3656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":257,"threads_started":1}
W20260806 19:43:24.022331 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:24.012658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:37109 (local address 127.21.202.126:42403)
0806 19:43:24.012954 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:24.012962 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:24.013102 (+ 140us) server_negotiation.cc:407] Connection header received
0806 19:43:24.013243 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:24.013247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:24.013311 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:24.013396 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:24.014316 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:24.015041 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:24.016492 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:24.017055 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:24.017161 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:24.022131 (+ 4970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:37109: BlockingRecv error: recv got EOF from 127.21.202.66:37109 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 06 19:43:24 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:24 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.67@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:24.494143 31770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:24.485604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.67:34477)
0806 19:43:24.486162 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:24.486179 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:24.486350 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:24.486735 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:24.486742 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:24.487143 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:24.487336 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:24.487346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:24.488394 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:24.488400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:24.489373 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:24.489385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:24.490114 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:24.490119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:24.490288 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:24.490963 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:24.490981 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:24.493782 (+ 2801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":280,"threads_started":1}
W20260806 19:43:24.494270 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:24.485728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:34477 (local address 127.21.202.126:42403)
0806 19:43:24.485912 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:24.485918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:24.486269 (+ 351us) server_negotiation.cc:407] Connection header received
0806 19:43:24.486516 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:24.486521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:24.486586 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:24.486689 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:24.487478 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:24.488239 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:24.489519 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:24.489968 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:24.490104 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:24.494019 (+ 3915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.67:34477: BlockingRecv error: recv got EOF from 127.21.202.67:34477 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 19:43:24 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:24 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.65@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:24.534000 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:24.524762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.65:53083)
0806 19:43:24.525441 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:24.525458 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:24.525620 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:24.525990 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:24.525995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:24.526436 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:24.526626 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:24.526636 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:24.527716 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:24.527724 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:24.528717 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:24.528731 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:24.529469 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:24.529475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:24.529663 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:24.530394 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:24.530413 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:24.533595 (+ 3182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":420,"threads_started":1}
W20260806 19:43:24.534102 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:24.524910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:53083 (local address 127.21.202.126:42403)
0806 19:43:24.525260 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:24.525268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:24.525549 (+ 281us) server_negotiation.cc:407] Connection header received
0806 19:43:24.525753 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:24.525758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:24.525845 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:24.525948 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:24.526769 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:24.527564 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:24.528876 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:24.529299 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:24.529450 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:24.533845 (+ 4395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.65:53083: BlockingRecv error: recv got EOF from 127.21.202.65:53083 (error 108)
Metrics: {"server-negotiator.queue_time_us":181}
Aug 06 19:43:25 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:43:25 dist-test-slave-77v9 krb5kdc[30708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.202.66@KRBTEST.COM for oryx/127.21.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:43:25.032846 31772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:25.023426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42403 (local address 127.21.202.66:38057)
0806 19:43:25.024034 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:25.024050 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:25.024228 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:25.024596 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:25.024601 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:25.025116 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:25.025288 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:25.025296 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:25.026369 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:25.026376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:25.027325 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:25.027336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:25.028139 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:25.028145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:25.028322 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:25.028991 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:25.029008 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:25.032476 (+ 3468us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.202.126:42403: Server oryx/127.21.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":314,"threads_started":1}
W20260806 19:43:25.033245 31457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:25.023575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:38057 (local address 127.21.202.126:42403)
0806 19:43:25.023820 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:25.023827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:25.024146 (+ 319us) server_negotiation.cc:407] Connection header received
0806 19:43:25.024359 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:25.024364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:25.024454 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:25.024559 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:25.025460 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:25.026219 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:25.027473 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:25.027978 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:25.028127 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:25.032981 (+ 4854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.202.66:38057: BlockingRecv error: recv got EOF from 127.21.202.66:38057 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 19:43:25.316026 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 31307
I20260806 19:43:25.329092 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:25.317279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46584 (local address 127.21.202.65:39763)
0806 19:43:25.317884 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:25.317892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:25.317921 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:43:25.317986 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:25.317992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:25.318253 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:25.318449 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:25.319936 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:25.321128 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:25.322438 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:25.323082 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:25.323200 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:25.325755 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:25.325763 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:25.325777 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:25.325811 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:25.328428 (+ 2617us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.202.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 19:43:25.328719 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46584: Request ticket server kudu/127.21.202.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":278,"threads_started":1}
W20260806 19:43:25.329246 31774 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46584: Request ticket server kudu/127.21.202.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 19:43:25.329234 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 31307: Not authorized: Client connection negotiation failed: client connection to 127.21.202.65:39763: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.202.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 19:43:25.329483 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 31307
I20260806 19:43:25.348543 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 31459
I20260806 19:43:25.362782 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:25.349831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46556 (local address 127.21.202.66:36733)
0806 19:43:25.350435 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:25.350443 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:25.350473 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:43:25.350534 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:25.350541 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:25.350763 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:25.350931 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:25.353162 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:25.354298 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:25.355630 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:25.356281 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:25.356388 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:25.359193 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:25.359202 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:25.359216 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:25.359275 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:25.362223 (+ 2948us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.202.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 19:43:25.362489 (+ 266us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46556: Request ticket server kudu/127.21.202.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":275,"threads_started":1}
W20260806 19:43:25.362876 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 31459: Not authorized: Client connection negotiation failed: client connection to 127.21.202.66:36733: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.202.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 19:43:25.363026 31775 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46556: Request ticket server kudu/127.21.202.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 19:43:25.363055 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 31459
I20260806 19:43:25.381273 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 31593
I20260806 19:43:25.394421 31776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:25.382286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47776 (local address 127.21.202.67:45983)
0806 19:43:25.382909 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:25.382916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:25.382946 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:43:25.383008 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:25.383014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:25.383234 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:25.383382 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:25.384771 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:25.385893 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:25.387300 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:25.387897 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:25.388009 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:25.390892 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:25.390902 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:25.390916 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:25.390955 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:25.393831 (+ 2876us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.202.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 19:43:25.394122 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47776: Request ticket server kudu/127.21.202.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":328,"threads_started":1}
W20260806 19:43:25.394582 31776 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47776: Request ticket server kudu/127.21.202.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 19:43:25.394598 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 31593: Not authorized: Client connection negotiation failed: client connection to 127.21.202.67:45983: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.202.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 19:43:25.394765 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 31593
I20260806 19:43:25.414213 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 31237
I20260806 19:43:25.415756 31300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:25.540407 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 31237
2026-08-06T19:43:25Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21616 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T19:43:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:43:25Z Disabled control of system clock
I20260806 19:43:25.636439 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:39699
--webserver_interface=127.21.202.126
--webserver_port=0
--builtin_ntp_servers=127.21.202.84:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:39699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 19:43:25.985678 31781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:25.985980 31781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:25.986064 31781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:25.993350 31781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:43:25.993458 31781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:25.993494 31781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:25.993544 31781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:43:25.993589 31781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:43:26.002796 31781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:39699
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:39699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:26.004122 31781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:26.005906 31781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:26.018483 31787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:26.018540 31789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:26.018805 31786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:26.019284 31781 server_base.cc:1034] running on GCE node
I20260806 19:43:26.020082 31781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:26.021615 31781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:26.022853 31781 hybrid_clock.cc:648] HybridClock initialized: now 1786045406022821 us; error 44 us; skew 500 ppm
I20260806 19:43:26.023242 31781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:43:26.029902 31781 webserver.cc:460] Webserver started at http://127.21.202.126:41465/ using document root <none> and password file <none>
I20260806 19:43:26.030767 31781 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:26.030915 31781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:26.031266 31781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:26.034456 31781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "0619ecd30953400dae3a5cd0961f6e71"
format_stamp: "Formatted at 2026-08-06 19:43:26 on dist-test-slave-77v9"
server_key: "b9020634656f8b7f0ed62d67e2b8b593"
server_key_iv: "9da4d957a27d8449a47a104d11850370"
server_key_version: "encryptionkey@0"
I20260806 19:43:26.035292 31781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "0619ecd30953400dae3a5cd0961f6e71"
format_stamp: "Formatted at 2026-08-06 19:43:26 on dist-test-slave-77v9"
server_key: "b9020634656f8b7f0ed62d67e2b8b593"
server_key_iv: "9da4d957a27d8449a47a104d11850370"
server_key_version: "encryptionkey@0"
I20260806 19:43:26.040688 31781 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 19:43:26.044312 31796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:26.045748 31781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:43:26.046101 31781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "0619ecd30953400dae3a5cd0961f6e71"
format_stamp: "Formatted at 2026-08-06 19:43:26 on dist-test-slave-77v9"
server_key: "b9020634656f8b7f0ed62d67e2b8b593"
server_key_iv: "9da4d957a27d8449a47a104d11850370"
server_key_version: "encryptionkey@0"
I20260806 19:43:26.046476 31781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:26.062204 31781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:26.063061 31781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:26.063364 31781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:26.082921 31781 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:39699
I20260806 19:43:26.082964 31847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:39699 every 8 connection(s)
I20260806 19:43:26.084342 31781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:43:26.088267 31848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:26.094300 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 31781
I20260806 19:43:26.094550 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:43:26.095407 22313 external_mini_cluster.cc:1428] Setting key 93282c1e4f45a15524fc074dc8929fb9
I20260806 19:43:26.099943 31848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71: Bootstrap starting.
I20260806 19:43:26.103137 31848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:26.104197 31848 log.cc:824] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71: Log is configured to *not* fsync() on all Append() calls
I20260806 19:43:26.107401 31848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71: No bootstrap required, opened a new log
I20260806 19:43:26.110559 31851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:26.097642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.21.202.126:39699)
0806 19:43:26.098750 (+ 1108us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:26.098763 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:26.098810 (+ 47us) server_negotiation.cc:407] Connection header received
0806 19:43:26.100682 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:26.100710 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:26.101240 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:26.101667 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:26.102414 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:26.103855 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:26.105807 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:26.106546 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:26.106782 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:26.108155 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:26.108166 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:26.108182 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:26.108716 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:26.108790 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:26.108876 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:26.109025 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:26.109258 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 19:43:26.109571 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":382,"threads_started":1}
I20260806 19:43:26.115403 31848 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0619ecd30953400dae3a5cd0961f6e71" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39699 } }
I20260806 19:43:26.115760 31848 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:26.115891 31848 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0619ecd30953400dae3a5cd0961f6e71, State: Initialized, Role: FOLLOWER
I20260806 19:43:26.116453 31848 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0619ecd30953400dae3a5cd0961f6e71" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39699 } }
I20260806 19:43:26.116658 31848 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:43:26.116833 31848 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:43:26.117031 31848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:26.119626 31848 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0619ecd30953400dae3a5cd0961f6e71" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39699 } }
I20260806 19:43:26.120223 31848 leader_election.cc:302] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0619ecd30953400dae3a5cd0961f6e71; no voters:
I20260806 19:43:26.120853 31848 leader_election.cc:288] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:43:26.121039 31853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:43:26.121740 31853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [term 1 LEADER]: Becoming Leader. State: Replica: 0619ecd30953400dae3a5cd0961f6e71, State: Running, Role: LEADER
I20260806 19:43:26.122269 31853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0619ecd30953400dae3a5cd0961f6e71" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39699 } }
I20260806 19:43:26.123025 31848 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:43:26.124966 31854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0619ecd30953400dae3a5cd0961f6e71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0619ecd30953400dae3a5cd0961f6e71" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39699 } } }
I20260806 19:43:26.125473 31854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [sys.catalog]: This master's current role is: LEADER
I20260806 19:43:26.125680 31855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0619ecd30953400dae3a5cd0961f6e71. Latest consensus state: current_term: 1 leader_uuid: "0619ecd30953400dae3a5cd0961f6e71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0619ecd30953400dae3a5cd0961f6e71" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39699 } } }
I20260806 19:43:26.126120 31855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71 [sys.catalog]: This master's current role is: LEADER
I20260806 19:43:26.130517 31858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:43:26.136670 31858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:43:26.146211 31858 catalog_manager.cc:1347] Generated new cluster ID: 6b32365e07cb4d9dbf14e08059188de4
I20260806 19:43:26.146373 31858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:43:26.167138 31858 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:43:26.169204 31858 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:43:26.179168 31858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0619ecd30953400dae3a5cd0961f6e71: Generated new TSK 0
I20260806 19:43:26.180714 31858 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:43:26.200052 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:39699
--builtin_ntp_servers=127.21.202.84:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 19:43:26.560855 31872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:26.561168 31872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:26.561252 31872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:26.569125 31872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:26.569247 31872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:26.569355 31872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:43:26.579504 31872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:39699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:26.581019 31872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:26.582861 31872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:26.597041 31881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:26.597008 31878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:26.597008 31879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:26.598403 31872 server_base.cc:1034] running on GCE node
I20260806 19:43:26.599153 31872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:26.600204 31872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:26.601471 31872 hybrid_clock.cc:648] HybridClock initialized: now 1786045406601439 us; error 37 us; skew 500 ppm
I20260806 19:43:26.601831 31872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:43:26.604290 31872 webserver.cc:460] Webserver started at http://127.21.202.65:38099/ using document root <none> and password file <none>
I20260806 19:43:26.605118 31872 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:26.605257 31872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:26.605620 31872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:26.608582 31872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "2c5c07bd6b634c3bb6ee0fc5e3a46f48"
format_stamp: "Formatted at 2026-08-06 19:43:26 on dist-test-slave-77v9"
server_key: "4036ade2e3d73f646ed9d27851af644c"
server_key_iv: "09b5b3b0bef7528d0257907a495db22e"
server_key_version: "encryptionkey@0"
I20260806 19:43:26.609458 31872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "2c5c07bd6b634c3bb6ee0fc5e3a46f48"
format_stamp: "Formatted at 2026-08-06 19:43:26 on dist-test-slave-77v9"
server_key: "4036ade2e3d73f646ed9d27851af644c"
server_key_iv: "09b5b3b0bef7528d0257907a495db22e"
server_key_version: "encryptionkey@0"
I20260806 19:43:26.615021 31872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 19:43:26.618773 31887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:26.620340 31872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:43:26.620499 31872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "2c5c07bd6b634c3bb6ee0fc5e3a46f48"
format_stamp: "Formatted at 2026-08-06 19:43:26 on dist-test-slave-77v9"
server_key: "4036ade2e3d73f646ed9d27851af644c"
server_key_iv: "09b5b3b0bef7528d0257907a495db22e"
server_key_version: "encryptionkey@0"
I20260806 19:43:26.620700 31872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:26.634430 31872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:26.635365 31872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:26.635667 31872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:26.636650 31872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:26.638342 31872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:26.638448 31872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:26.638535 31872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:26.638595 31872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:26.676200 31872 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:35881
I20260806 19:43:26.676256 32000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:35881 every 8 connection(s)
I20260806 19:43:26.677605 31872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:43:26.687712 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 31872
I20260806 19:43:26.687985 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:43:26.688705 22313 external_mini_cluster.cc:1428] Setting key 6a1c87c8c9fd154e44f3f8527b854e66
I20260806 19:43:26.690716 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:26.680522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:37803 (local address 127.21.202.126:39699)
0806 19:43:26.681058 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:26.681063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:26.682212 (+ 1149us) server_negotiation.cc:407] Connection header received
0806 19:43:26.684000 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:26.684006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:26.684099 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:26.684225 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:26.685406 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:26.686147 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:26.687721 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:26.688245 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:26.688377 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:26.689817 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:26.689823 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:26.689828 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:26.689853 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:26.689893 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:26.689906 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:26.690005 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:26.690242 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 19:43:26.690342 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":281,"threads_started":1}
I20260806 19:43:26.691706 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:39699
--builtin_ntp_servers=127.21.202.84:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 19:43:26.691740 32004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:26.681151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:39699 (local address 127.21.202.65:37803)
0806 19:43:26.682048 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:26.682085 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:26.683720 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:26.684479 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:26.684488 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:26.684574 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:26.685235 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:26.685253 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:26.686314 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:26.686318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:26.687555 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:26.687568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:26.688480 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:26.688489 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:26.688707 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:26.689489 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:43:26.689491 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:26.689534 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:43:26.689612 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:43:26.690007 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:43:26.690023 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:43:26.690032 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 19:43:26.690214 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 19:43:26.690636 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":286,"threads_started":1}
I20260806 19:43:26.693301 32001 heartbeater.cc:347] Connected to a master server at 127.21.202.126:39699
I20260806 19:43:26.693671 32001 heartbeater.cc:464] Registering TS with master...
I20260806 19:43:26.694823 32001 heartbeater.cc:511] Master 127.21.202.126:39699 requested a full tablet report, sending...
I20260806 19:43:26.697141 31813 ts_manager.cc:194] Registered new tserver with Master: 2c5c07bd6b634c3bb6ee0fc5e3a46f48 (127.21.202.65:35881)
I20260806 19:43:26.698604 31813 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.202.65:37803
W20260806 19:43:27.049836 32005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:27.050123 32005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:27.050257 32005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:27.057508 32005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:27.057610 32005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:27.057775 32005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:43:27.066854 32005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:39699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:27.068166 32005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:27.070056 32005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:27.083796 32013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:27.083846 32011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:27.084203 32010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:27.084666 32005 server_base.cc:1034] running on GCE node
I20260806 19:43:27.085359 32005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:27.086336 32005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:27.087605 32005 hybrid_clock.cc:648] HybridClock initialized: now 1786045407087513 us; error 52 us; skew 500 ppm
I20260806 19:43:27.087929 32005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:43:27.090236 32005 webserver.cc:460] Webserver started at http://127.21.202.66:32773/ using document root <none> and password file <none>
I20260806 19:43:27.090950 32005 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:27.091082 32005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:27.091473 32005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:27.094322 32005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "5253f8266a8f474ab434200b8aeb019e"
format_stamp: "Formatted at 2026-08-06 19:43:27 on dist-test-slave-77v9"
server_key: "cf2ff74d3e0b413ba7f486b7280f21ec"
server_key_iv: "acf06a68b47f9fca1638136b41e1c71d"
server_key_version: "encryptionkey@0"
I20260806 19:43:27.095108 32005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "5253f8266a8f474ab434200b8aeb019e"
format_stamp: "Formatted at 2026-08-06 19:43:27 on dist-test-slave-77v9"
server_key: "cf2ff74d3e0b413ba7f486b7280f21ec"
server_key_iv: "acf06a68b47f9fca1638136b41e1c71d"
server_key_version: "encryptionkey@0"
I20260806 19:43:27.100243 32005 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 19:43:27.103889 32020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:27.105283 32005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:43:27.105473 32005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "5253f8266a8f474ab434200b8aeb019e"
format_stamp: "Formatted at 2026-08-06 19:43:27 on dist-test-slave-77v9"
server_key: "cf2ff74d3e0b413ba7f486b7280f21ec"
server_key_iv: "acf06a68b47f9fca1638136b41e1c71d"
server_key_version: "encryptionkey@0"
I20260806 19:43:27.105698 32005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:27.129454 32005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:27.130416 32005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:27.130741 32005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:27.131839 32005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:27.133409 32005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:27.133529 32005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:27.133659 32005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:27.133751 32005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:27.169670 32005 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:34939
I20260806 19:43:27.169729 32133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:34939 every 8 connection(s)
I20260806 19:43:27.170980 32005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:43:27.173259 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 32005
I20260806 19:43:27.173480 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:43:27.174103 22313 external_mini_cluster.cc:1428] Setting key e505dd6714216b118ddeac9d02250bc6
I20260806 19:43:27.177460 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:39699
--builtin_ntp_servers=127.21.202.84:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 19:43:27.184638 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:27.173806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:42399 (local address 127.21.202.126:39699)
0806 19:43:27.174242 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:27.174248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:27.175821 (+ 1573us) server_negotiation.cc:407] Connection header received
0806 19:43:27.178015 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:27.178021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:27.178115 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:27.178229 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:27.179284 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.180146 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.181826 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.182241 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.182379 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:27.183795 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:27.183800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:27.183803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:27.183822 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:27.183853 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:27.183865 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:27.183992 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:27.184223 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 19:43:27.184324 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275}
I20260806 19:43:27.185727 32136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:27.174454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:39699 (local address 127.21.202.66:42399)
0806 19:43:27.175609 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:27.175673 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:27.177750 (+ 2077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:27.178457 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:27.178466 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:27.178558 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:27.179115 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:27.179132 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.180317 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.180321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:27.181668 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:27.181681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.182478 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.182486 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:27.182711 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:27.183459 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:43:27.183461 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:27.183501 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:43:27.183583 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:43:27.184001 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:43:27.184014 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:43:27.184024 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 19:43:27.184212 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 19:43:27.184631 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":410,"threads_started":1}
I20260806 19:43:27.187290 32134 heartbeater.cc:347] Connected to a master server at 127.21.202.126:39699
I20260806 19:43:27.187644 32134 heartbeater.cc:464] Registering TS with master...
I20260806 19:43:27.188639 32134 heartbeater.cc:511] Master 127.21.202.126:39699 requested a full tablet report, sending...
I20260806 19:43:27.190436 31813 ts_manager.cc:194] Registered new tserver with Master: 5253f8266a8f474ab434200b8aeb019e (127.21.202.66:34939)
I20260806 19:43:27.191289 31813 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.202.66:42399
W20260806 19:43:27.529716 32137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:27.530083 32137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:27.530172 32137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:27.537777 32137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:27.537886 32137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:27.538000 32137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:43:27.547170 32137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:39699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:27.548491 32137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:27.550366 32137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:27.564548 32142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:27.564504 32143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:27.564462 32145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:27.565342 32137 server_base.cc:1034] running on GCE node
I20260806 19:43:27.566102 32137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:27.567198 32137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:27.568449 32137 hybrid_clock.cc:648] HybridClock initialized: now 1786045407568377 us; error 68 us; skew 500 ppm
I20260806 19:43:27.568790 32137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:43:27.571331 32137 webserver.cc:460] Webserver started at http://127.21.202.67:40001/ using document root <none> and password file <none>
I20260806 19:43:27.572079 32137 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:27.572223 32137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:27.572568 32137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:27.576061 32137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "98c4d92434e64e27b46b1697f4d5b991"
format_stamp: "Formatted at 2026-08-06 19:43:27 on dist-test-slave-77v9"
server_key: "1c95d2c98c21755b0de36be11dc1d147"
server_key_iv: "7929bdfbfd4a66836445af06819103e2"
server_key_version: "encryptionkey@0"
I20260806 19:43:27.576967 32137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "98c4d92434e64e27b46b1697f4d5b991"
format_stamp: "Formatted at 2026-08-06 19:43:27 on dist-test-slave-77v9"
server_key: "1c95d2c98c21755b0de36be11dc1d147"
server_key_iv: "7929bdfbfd4a66836445af06819103e2"
server_key_version: "encryptionkey@0"
I20260806 19:43:27.582512 32137 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 19:43:27.586084 32151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:27.587576 32137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 19:43:27.587759 32137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "98c4d92434e64e27b46b1697f4d5b991"
format_stamp: "Formatted at 2026-08-06 19:43:27 on dist-test-slave-77v9"
server_key: "1c95d2c98c21755b0de36be11dc1d147"
server_key_iv: "7929bdfbfd4a66836445af06819103e2"
server_key_version: "encryptionkey@0"
I20260806 19:43:27.588014 32137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:27.625106 32137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:27.626329 32137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:27.626708 32137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:27.627810 32137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:27.629594 32137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:27.629750 32137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:27.629926 32137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:27.630028 32137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:27.670334 32137 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:38899
I20260806 19:43:27.670440 32267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:38899 every 8 connection(s)
I20260806 19:43:27.671696 32137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:43:27.679503 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 32137
I20260806 19:43:27.679715 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:43:27.680269 22313 external_mini_cluster.cc:1428] Setting key 36bff8e3a60b5f7127c941cb37ebfb6d
I20260806 19:43:27.686324 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:27.674616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:33019 (local address 127.21.202.126:39699)
0806 19:43:27.674805 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:27.674810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:27.676500 (+ 1690us) server_negotiation.cc:407] Connection header received
0806 19:43:27.678626 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:27.678636 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:27.678735 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:27.678870 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:27.680237 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.681278 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.683022 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.683582 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.683719 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:27.685335 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:27.685341 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:27.685344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:27.685369 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:27.685405 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:27.685419 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:27.685528 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:27.685779 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 19:43:27.685895 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 19:43:27.687469 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:27.675084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:39699 (local address 127.21.202.67:33019)
0806 19:43:27.676282 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:27.676337 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:27.678251 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:27.679140 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:27.679150 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:27.679239 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:27.680050 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:27.680072 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.681491 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.681496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:27.682872 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:27.682885 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.683826 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.683833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:27.684059 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:27.684765 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:43:27.684768 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:27.684971 (+ 203us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:43:27.685100 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:43:27.685531 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:43:27.685546 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:43:27.685556 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 19:43:27.685765 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 19:43:27.686244 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":418,"threads_started":1}
I20260806 19:43:27.689019 32268 heartbeater.cc:347] Connected to a master server at 127.21.202.126:39699
I20260806 19:43:27.689340 32268 heartbeater.cc:464] Registering TS with master...
I20260806 19:43:27.690100 32268 heartbeater.cc:511] Master 127.21.202.126:39699 requested a full tablet report, sending...
I20260806 19:43:27.691660 31813 ts_manager.cc:194] Registered new tserver with Master: 98c4d92434e64e27b46b1697f4d5b991 (127.21.202.67:38899)
I20260806 19:43:27.692574 31813 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.202.67:33019
I20260806 19:43:27.696002 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:43:27.702386 32001 heartbeater.cc:503] Master 127.21.202.126:39699 was elected leader, sending a full tablet report...
W20260806 19:43:27.702734 32003 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:27.700994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56790 (local address 127.21.202.126:39699)
0806 19:43:27.701174 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:27.701179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:27.701686 (+ 507us) server_negotiation.cc:407] Connection header received
0806 19:43:27.701828 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:27.701832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:27.701893 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:27.701991 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:27.702491 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56790: BlockingRecv error: recv got EOF from 127.0.0.1:56790 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 19:43:27.704305 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 31872
I20260806 19:43:27.712911 32277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:27.705527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.21.202.65:35881)
0806 19:43:27.706074 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:27.706096 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:27.706122 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:43:27.706176 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:27.706182 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:27.706402 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:27.706556 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:27.706954 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.708066 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.709450 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.710171 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.710279 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:27.711411 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:27.711419 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:27.711432 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:27.712034 (+ 602us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:27.712095 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:27.712115 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:27.712210 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:27.712346 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 19:43:27.712496 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":278,"threads_started":1}
I20260806 19:43:27.714140 31996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:27.797151 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 31872
I20260806 19:43:27.815272 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 32005
I20260806 19:43:27.823633 32279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:27.816208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59222 (local address 127.21.202.66:34939)
0806 19:43:27.816772 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:27.816779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:27.816857 (+ 78us) server_negotiation.cc:407] Connection header received
0806 19:43:27.816936 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:27.816943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:27.817169 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:27.817323 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:27.817767 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.818811 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.820141 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.820726 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.821069 (+ 343us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:27.821999 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:27.822014 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:27.822032 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:27.822713 (+ 681us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:27.822764 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:27.822785 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:27.822896 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:27.823034 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 19:43:27.823170 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":287,"threads_started":1}
I20260806 19:43:27.825103 32129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:27.907773 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 32005
I20260806 19:43:27.926926 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 32137
I20260806 19:43:27.935684 32281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:27.927979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45128 (local address 127.21.202.67:38899)
0806 19:43:27.928660 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:27.928668 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:27.928699 (+ 31us) server_negotiation.cc:407] Connection header received
0806 19:43:27.928761 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:27.928767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:27.929096 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:27.929230 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:27.929690 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.930824 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.932337 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:27.933037 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:27.933138 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:27.934198 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:27.934207 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:27.934219 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:27.934812 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:27.934861 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:27.934880 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:27.934978 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:27.935152 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 19:43:27.935291 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":363,"threads_started":1}
I20260806 19:43:27.937140 32263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:28.024223 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 32137
I20260806 19:43:28.043908 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 31781
I20260806 19:43:28.045261 31843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:28.150902 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 31781
2026-08-06T19:43:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2600 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T19:43:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:43:28Z Disabled control of system clock
I20260806 19:43:28.233227 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:43689
--webserver_interface=127.21.202.126
--webserver_port=0
--builtin_ntp_servers=127.21.202.84:46341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:43689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 19:43:28.576298 32287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:28.576608 32287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:28.576792 32287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:28.584316 32287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:43:28.584430 32287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:28.584504 32287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:28.584619 32287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:43:28.584681 32287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:43:28.594460 32287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:46341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:43689
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:43689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:28.595830 32287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:28.597776 32287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:28.609858 32292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:28.609726 32295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:28.609790 32293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:28.611042 32287 server_base.cc:1034] running on GCE node
I20260806 19:43:28.611825 32287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:28.613408 32287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:28.614614 32287 hybrid_clock.cc:648] HybridClock initialized: now 1786045408614574 us; error 33 us; skew 500 ppm
I20260806 19:43:28.614974 32287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:43:28.617528 32287 webserver.cc:460] Webserver started at http://127.21.202.126:39639/ using document root <none> and password file <none>
I20260806 19:43:28.618265 32287 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:28.618360 32287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:28.618738 32287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:28.621785 32287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "594838c1c57d4a39a32a49cb8c3afa47"
format_stamp: "Formatted at 2026-08-06 19:43:28 on dist-test-slave-77v9"
server_key: "b29743187eb8b9901e5b8d04e3ff6544"
server_key_iv: "e46a5daae8fee84a61a2af1b9dc2c3ea"
server_key_version: "encryptionkey@0"
I20260806 19:43:28.622679 32287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "594838c1c57d4a39a32a49cb8c3afa47"
format_stamp: "Formatted at 2026-08-06 19:43:28 on dist-test-slave-77v9"
server_key: "b29743187eb8b9901e5b8d04e3ff6544"
server_key_iv: "e46a5daae8fee84a61a2af1b9dc2c3ea"
server_key_version: "encryptionkey@0"
I20260806 19:43:28.628165 32287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:43:28.631934 32302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:28.633795 32287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:43:28.634019 32287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "594838c1c57d4a39a32a49cb8c3afa47"
format_stamp: "Formatted at 2026-08-06 19:43:28 on dist-test-slave-77v9"
server_key: "b29743187eb8b9901e5b8d04e3ff6544"
server_key_iv: "e46a5daae8fee84a61a2af1b9dc2c3ea"
server_key_version: "encryptionkey@0"
I20260806 19:43:28.634286 32287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:28.641083 32287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:28.641966 32287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:28.642316 32287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:28.663393 32287 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:43689
I20260806 19:43:28.663426 32354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:43689 every 8 connection(s)
I20260806 19:43:28.664973 32287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:43:28.668885 32355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:28.670593 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 32287
I20260806 19:43:28.670799 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:43:28.671645 22313 external_mini_cluster.cc:1428] Setting key 98bd6932549293ba3471a72ec9d54f6e
I20260806 19:43:28.681296 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:28.674122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49038 (local address 127.21.202.126:43689)
0806 19:43:28.675292 (+ 1170us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:28.675311 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:28.675362 (+ 51us) server_negotiation.cc:407] Connection header received
0806 19:43:28.677278 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:28.677302 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:28.677732 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:28.678178 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:28.679443 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:28.679453 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:28.679470 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:28.679485 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:28.679546 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:28.679570 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:28.679875 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:28.680021 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 19:43:28.680272 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":374,"threads_started":1}
I20260806 19:43:28.681735 32355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47: Bootstrap starting.
I20260806 19:43:28.684923 32355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:28.686312 32355 log.cc:824] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47: Log is configured to *not* fsync() on all Append() calls
I20260806 19:43:28.689622 32355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47: No bootstrap required, opened a new log
I20260806 19:43:28.697340 32355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594838c1c57d4a39a32a49cb8c3afa47" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 43689 } }
I20260806 19:43:28.697748 32355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:28.697857 32355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 594838c1c57d4a39a32a49cb8c3afa47, State: Initialized, Role: FOLLOWER
I20260806 19:43:28.698441 32355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594838c1c57d4a39a32a49cb8c3afa47" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 43689 } }
I20260806 19:43:28.698695 32355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:43:28.698839 32355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:43:28.699018 32355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:28.701647 32355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594838c1c57d4a39a32a49cb8c3afa47" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 43689 } }
I20260806 19:43:28.702194 32355 leader_election.cc:302] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 594838c1c57d4a39a32a49cb8c3afa47; no voters:
I20260806 19:43:28.702795 32355 leader_election.cc:288] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:43:28.702961 32360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:43:28.703787 32360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [term 1 LEADER]: Becoming Leader. State: Replica: 594838c1c57d4a39a32a49cb8c3afa47, State: Running, Role: LEADER
I20260806 19:43:28.704386 32360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594838c1c57d4a39a32a49cb8c3afa47" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 43689 } }
I20260806 19:43:28.705129 32355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:43:28.707474 32362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 594838c1c57d4a39a32a49cb8c3afa47. Latest consensus state: current_term: 1 leader_uuid: "594838c1c57d4a39a32a49cb8c3afa47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594838c1c57d4a39a32a49cb8c3afa47" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 43689 } } }
I20260806 19:43:28.707684 32361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "594838c1c57d4a39a32a49cb8c3afa47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594838c1c57d4a39a32a49cb8c3afa47" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 43689 } } }
I20260806 19:43:28.707944 32362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [sys.catalog]: This master's current role is: LEADER
I20260806 19:43:28.708025 32361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47 [sys.catalog]: This master's current role is: LEADER
I20260806 19:43:28.709857 32369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:43:28.714339 32369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:43:28.723933 32369 catalog_manager.cc:1347] Generated new cluster ID: 67d0b087edeb494eb807345361b276bb
I20260806 19:43:28.724120 32369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:43:28.748210 32369 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:43:28.748963 32369 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:43:28.759644 32369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 594838c1c57d4a39a32a49cb8c3afa47: Generated new TSK 0
I20260806 19:43:28.760872 32369 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:43:28.778877 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:43689
--builtin_ntp_servers=127.21.202.84:46341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 19:43:29.134039 32379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:29.134415 32379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:29.134591 32379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:29.142912 32379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:29.143085 32379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:29.143278 32379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:43:29.153632 32379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:46341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:43689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:29.155088 32379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:29.156986 32379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:29.170612 32384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:29.170641 32385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:29.170755 32387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:29.171984 32379 server_base.cc:1034] running on GCE node
I20260806 19:43:29.172672 32379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:29.173759 32379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:29.175045 32379 hybrid_clock.cc:648] HybridClock initialized: now 1786045409174972 us; error 67 us; skew 500 ppm
I20260806 19:43:29.175426 32379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:43:29.178203 32379 webserver.cc:460] Webserver started at http://127.21.202.65:41707/ using document root <none> and password file <none>
I20260806 19:43:29.179294 32379 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:29.179399 32379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:29.179742 32379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:29.183094 32379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "c5303fa2f6054be691db50b2ebda6d92"
format_stamp: "Formatted at 2026-08-06 19:43:29 on dist-test-slave-77v9"
server_key: "d925e8874763786489d41b2a096d724f"
server_key_iv: "e03f8db97f6570348bd214d39d866d5f"
server_key_version: "encryptionkey@0"
I20260806 19:43:29.183952 32379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "c5303fa2f6054be691db50b2ebda6d92"
format_stamp: "Formatted at 2026-08-06 19:43:29 on dist-test-slave-77v9"
server_key: "d925e8874763786489d41b2a096d724f"
server_key_iv: "e03f8db97f6570348bd214d39d866d5f"
server_key_version: "encryptionkey@0"
I20260806 19:43:29.189622 32379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 19:43:29.193342 32393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:29.194911 32379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:43:29.195119 32379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "c5303fa2f6054be691db50b2ebda6d92"
format_stamp: "Formatted at 2026-08-06 19:43:29 on dist-test-slave-77v9"
server_key: "d925e8874763786489d41b2a096d724f"
server_key_iv: "e03f8db97f6570348bd214d39d866d5f"
server_key_version: "encryptionkey@0"
I20260806 19:43:29.195370 32379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:29.201628 32379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:29.202479 32379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:29.202759 32379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:29.203775 32379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:29.205487 32379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:29.205632 32379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:29.205782 32379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:29.205873 32379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:29.242568 32379 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:44593
I20260806 19:43:29.242614 32508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:44593 every 8 connection(s)
I20260806 19:43:29.243913 32379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:43:29.250778 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 32379
I20260806 19:43:29.250991 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:43:29.251914 22313 external_mini_cluster.cc:1428] Setting key f30fc2ad6d49524ea3fe310023475865
I20260806 19:43:29.252736 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:29.246651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:35761 (local address 127.21.202.126:43689)
0806 19:43:29.247158 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:29.247165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:29.248445 (+ 1280us) server_negotiation.cc:407] Connection header received
0806 19:43:29.250102 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:29.250106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:29.250159 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:29.250257 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:29.251872 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:29.251876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:29.251879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:29.251895 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:29.251953 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:29.251998 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:29.252160 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:29.252354 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 19:43:29.252424 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":253,"threads_started":1}
I20260806 19:43:29.253877 32512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:29.247309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:43689 (local address 127.21.202.65:35761)
0806 19:43:29.248278 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:29.248315 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:29.249865 (+ 1550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:29.250541 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:29.250554 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:29.250680 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:29.251473 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:43:29.251477 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:29.251541 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:43:29.251657 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:43:29.252120 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:43:29.252133 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:43:29.252143 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 19:43:29.252452 (+ 309us) client_negotiation.cc:240] Negotiation successful
0806 19:43:29.252698 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":299,"threads_started":1}
I20260806 19:43:29.255026 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:43689
--builtin_ntp_servers=127.21.202.84:46341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 19:43:29.255481 32509 heartbeater.cc:347] Connected to a master server at 127.21.202.126:43689
I20260806 19:43:29.255810 32509 heartbeater.cc:464] Registering TS with master...
I20260806 19:43:29.256539 32509 heartbeater.cc:511] Master 127.21.202.126:43689 requested a full tablet report, sending...
I20260806 19:43:29.258714 32319 ts_manager.cc:194] Registered new tserver with Master: c5303fa2f6054be691db50b2ebda6d92 (127.21.202.65:44593)
W20260806 19:43:29.604035 32513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:29.604331 32513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:29.604475 32513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:29.612500 32513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:29.612617 32513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:29.612798 32513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:43:29.622365 32513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:46341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:43689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:29.623745 32513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:29.625669 32513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:29.639058 32518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:29.638969 32521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:29.639071 32519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:29.640329 32513 server_base.cc:1034] running on GCE node
I20260806 19:43:29.641083 32513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:29.642067 32513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:29.643277 32513 hybrid_clock.cc:648] HybridClock initialized: now 1786045409643208 us; error 60 us; skew 500 ppm
I20260806 19:43:29.643616 32513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:43:29.646216 32513 webserver.cc:460] Webserver started at http://127.21.202.66:42519/ using document root <none> and password file <none>
I20260806 19:43:29.647123 32513 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:29.647222 32513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:29.647603 32513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:29.650784 32513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "126d5b9170a3437b968e1b63652808ab"
format_stamp: "Formatted at 2026-08-06 19:43:29 on dist-test-slave-77v9"
server_key: "cf2fc2babd9757bb5071a76beedcb9a9"
server_key_iv: "fd56d5bf393b6b9c20b99d14178475de"
server_key_version: "encryptionkey@0"
I20260806 19:43:29.651652 32513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "126d5b9170a3437b968e1b63652808ab"
format_stamp: "Formatted at 2026-08-06 19:43:29 on dist-test-slave-77v9"
server_key: "cf2fc2babd9757bb5071a76beedcb9a9"
server_key_iv: "fd56d5bf393b6b9c20b99d14178475de"
server_key_version: "encryptionkey@0"
I20260806 19:43:29.657768 32513 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 19:43:29.661782 32527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:29.663493 32513 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:43:29.663676 32513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "126d5b9170a3437b968e1b63652808ab"
format_stamp: "Formatted at 2026-08-06 19:43:29 on dist-test-slave-77v9"
server_key: "cf2fc2babd9757bb5071a76beedcb9a9"
server_key_iv: "fd56d5bf393b6b9c20b99d14178475de"
server_key_version: "encryptionkey@0"
I20260806 19:43:29.664140 32513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:29.670485 32513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:29.671316 32513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:29.671605 32513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:29.672601 32513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:29.674207 32513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:29.674300 32513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:29.674465 32513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:29.674562 32513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:29.711711 32513 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:46023
I20260806 19:43:29.711874 32644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:46023 every 8 connection(s)
I20260806 19:43:29.713311 32513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:43:29.715097 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 32513
I20260806 19:43:29.715293 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:43:29.715925 22313 external_mini_cluster.cc:1428] Setting key e505e89097bd7d917a5b8d41c4f69383
I20260806 19:43:29.719530 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:43689
--builtin_ntp_servers=127.21.202.84:46341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 19:43:29.724592 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:29.716396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:37387 (local address 127.21.202.126:43689)
0806 19:43:29.716612 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:29.716620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:29.718481 (+ 1861us) server_negotiation.cc:407] Connection header received
0806 19:43:29.721900 (+ 3419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:29.721905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:29.721979 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:29.722091 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:29.723785 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:29.723791 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:29.723794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:29.723845 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:29.723897 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:29.723912 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:29.724033 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:29.724291 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 19:43:29.724353 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 19:43:29.726155 32647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:29.717149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:43689 (local address 127.21.202.66:37387)
0806 19:43:29.718285 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:29.718352 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:29.721493 (+ 3141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:29.722407 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:29.722423 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:29.722572 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:29.723365 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:43:29.723371 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:29.723431 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:43:29.723561 (+ 130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:43:29.724035 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:43:29.724049 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:43:29.724061 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 19:43:29.724380 (+ 319us) client_negotiation.cc:240] Negotiation successful
0806 19:43:29.724690 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":350,"threads_started":1}
I20260806 19:43:29.727900 32645 heartbeater.cc:347] Connected to a master server at 127.21.202.126:43689
I20260806 19:43:29.728296 32645 heartbeater.cc:464] Registering TS with master...
I20260806 19:43:29.729142 32645 heartbeater.cc:511] Master 127.21.202.126:43689 requested a full tablet report, sending...
I20260806 19:43:29.730926 32319 ts_manager.cc:194] Registered new tserver with Master: 126d5b9170a3437b968e1b63652808ab (127.21.202.66:46023)
W20260806 19:43:30.095448 32648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:30.095784 32648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:30.095887 32648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:30.103747 32648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:30.103884 32648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:30.104009 32648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:43:30.114820 32648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:46341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:43689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:30.116204 32648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:30.118224 32648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:30.132858 32653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:30.132790 32654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:30.132757 32657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:30.134022 32648 server_base.cc:1034] running on GCE node
I20260806 19:43:30.134693 32648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:30.135769 32648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:30.136996 32648 hybrid_clock.cc:648] HybridClock initialized: now 1786045410136933 us; error 55 us; skew 500 ppm
I20260806 19:43:30.137368 32648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:43:30.140120 32648 webserver.cc:460] Webserver started at http://127.21.202.67:45359/ using document root <none> and password file <none>
I20260806 19:43:30.140946 32648 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:30.141083 32648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:30.141436 32648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:30.144502 32648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "936f25821b7a4f9d8714f971b78eebf4"
format_stamp: "Formatted at 2026-08-06 19:43:30 on dist-test-slave-77v9"
server_key: "5f44e2c1e17eb2f5461408a816501203"
server_key_iv: "f6eed2e799d2ae59d01af66c78a0013d"
server_key_version: "encryptionkey@0"
I20260806 19:43:30.145510 32648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "936f25821b7a4f9d8714f971b78eebf4"
format_stamp: "Formatted at 2026-08-06 19:43:30 on dist-test-slave-77v9"
server_key: "5f44e2c1e17eb2f5461408a816501203"
server_key_iv: "f6eed2e799d2ae59d01af66c78a0013d"
server_key_version: "encryptionkey@0"
I20260806 19:43:30.151357 32648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 19:43:30.155287 32663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:30.156898 32648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:43:30.157114 32648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "936f25821b7a4f9d8714f971b78eebf4"
format_stamp: "Formatted at 2026-08-06 19:43:30 on dist-test-slave-77v9"
server_key: "5f44e2c1e17eb2f5461408a816501203"
server_key_iv: "f6eed2e799d2ae59d01af66c78a0013d"
server_key_version: "encryptionkey@0"
I20260806 19:43:30.157404 32648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:30.163952 32648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:30.165033 32648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:30.165330 32648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:30.166433 32648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:30.168056 32648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:30.168184 32648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:30.168324 32648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:30.168438 32648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:30.207378 32648 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:41203
I20260806 19:43:30.207439 311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:41203 every 8 connection(s)
I20260806 19:43:30.209023 32648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:43:30.213094 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 32648
I20260806 19:43:30.213289 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:43:30.213912 22313 external_mini_cluster.cc:1428] Setting key 756ec8ebcb5498df6c3e22823c7a3829
I20260806 19:43:30.219275 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:30.211897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:48081 (local address 127.21.202.126:43689)
0806 19:43:30.212152 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:30.212161 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:30.214109 (+ 1948us) server_negotiation.cc:407] Connection header received
0806 19:43:30.216285 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:30.216290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:30.216385 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:30.216505 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:30.218340 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:30.218345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:30.218349 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:30.218370 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:30.218409 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:30.218423 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:30.218556 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:30.218968 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 19:43:30.219027 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 19:43:30.220307 314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:30.212547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:43689 (local address 127.21.202.67:48081)
0806 19:43:30.213893 (+ 1346us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:30.213955 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:30.215960 (+ 2005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:30.216788 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:30.216798 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:30.216977 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:30.217925 (+ 948us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:43:30.217930 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:30.217999 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:43:30.218105 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:43:30.218704 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:43:30.218713 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:43:30.218720 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 19:43:30.218997 (+ 277us) client_negotiation.cc:240] Negotiation successful
0806 19:43:30.219230 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":428,"threads_started":1}
I20260806 19:43:30.222149 312 heartbeater.cc:347] Connected to a master server at 127.21.202.126:43689
I20260806 19:43:30.222492 312 heartbeater.cc:464] Registering TS with master...
I20260806 19:43:30.223214 312 heartbeater.cc:511] Master 127.21.202.126:43689 requested a full tablet report, sending...
I20260806 19:43:30.224876 32319 ts_manager.cc:194] Registered new tserver with Master: 936f25821b7a4f9d8714f971b78eebf4 (127.21.202.67:41203)
I20260806 19:43:30.231436 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 19:43:30.239192 32511 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:43:30.237255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60738 (local address 127.21.202.126:43689)
0806 19:43:30.237491 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:30.237500 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:30.238133 (+ 633us) server_negotiation.cc:407] Connection header received
0806 19:43:30.238286 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:30.238290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:30.238353 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:30.238438 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:30.238839 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60738: BlockingRecv error: recv got EOF from 127.0.0.1:60738 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 19:43:30.240166 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 32379
I20260806 19:43:30.244416 321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:30.241355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55046 (local address 127.21.202.65:44593)
0806 19:43:30.241972 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:30.241979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:30.242009 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:43:30.242131 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:30.242157 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:30.242415 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:30.242559 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:30.243685 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:30.243692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:30.243709 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:30.243723 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:30.243769 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:30.243787 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:30.243935 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:30.244038 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 19:43:30.244121 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":270,"threads_started":1}
I20260806 19:43:30.245875 32504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:30.333320 32509 heartbeater.cc:503] Master 127.21.202.126:43689 was elected leader, sending a full tablet report...
I20260806 19:43:30.335112 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 32379
I20260806 19:43:30.353300 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 32513
I20260806 19:43:30.357154 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:30.354293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58102 (local address 127.21.202.66:46023)
0806 19:43:30.354900 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:30.354907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:30.354937 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:43:30.355007 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:30.355013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:30.355232 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:30.355364 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:30.356378 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:30.356385 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:30.356397 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:30.356426 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:30.356484 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:30.356499 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:30.356608 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:30.356724 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 19:43:30.356860 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":322,"threads_started":1}
I20260806 19:43:30.358379 32640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:30.441946 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 32513
I20260806 19:43:30.460216 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 32648
I20260806 19:43:30.465502 325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:30.461725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56702 (local address 127.21.202.67:41203)
0806 19:43:30.462455 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:30.462462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:30.462493 (+ 31us) server_negotiation.cc:407] Connection header received
0806 19:43:30.462585 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:30.462594 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:30.462832 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:30.462976 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:30.464501 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:30.464509 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:30.464521 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:43:30.464546 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:30.464597 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:43:30.464613 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:30.464767 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:30.465007 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 19:43:30.465095 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":347,"threads_started":1}
I20260806 19:43:30.466840 306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:30.551997 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 32648
I20260806 19:43:30.570228 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 32287
I20260806 19:43:30.571640 32350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:30.676142 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 32287
2026-08-06T19:43:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2526 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:43:30 dist-test-slave-77v9 krb5kdc[330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:43:30 dist-test-slave-77v9 krb5kdc[330](info): set up 2 sockets
Aug 06 19:43:30 dist-test-slave-77v9 krb5kdc[330](info): commencing operation
krb5kdc: starting...
W20260806 19:43:31.842252 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.005s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:43:32 dist-test-slave-77v9 krb5kdc[330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:43:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:43:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:43:32.146486 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42629
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:42629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:43:32.482344 343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:32.482656 343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:32.482812 343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:32.490124 343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:43:32.490231 343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:32.490320 343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:32.490402 343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:43:32.490468 343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:43:32.500380 343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:42629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:32.501785 343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:32.503116 343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:32.516100 348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:32.516127 349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:32.516153 351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:32.517199 343 server_base.cc:1034] running on GCE node
I20260806 19:43:32.518349 343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:32.520349 343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:32.521660 343 hybrid_clock.cc:648] HybridClock initialized: now 1786045412521577 us; error 48 us; skew 500 ppm
I20260806 19:43:32.522133 343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:43:32 dist-test-slave-77v9 krb5kdc[330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045412, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:43:32.530678 343 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:43:32.532485 343 webserver.cc:460] Webserver started at http://127.21.202.126:45715/ using document root <none> and password file <none>
I20260806 19:43:32.533299 343 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:32.533464 343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:32.533803 343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:32.536989 343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "33d451c5a9c74f1cb09aeae96770b392"
format_stamp: "Formatted at 2026-08-06 19:43:32 on dist-test-slave-77v9"
server_key: "cbf1be3db17426790ea67cee5e2c4d66"
server_key_iv: "4f58f896116ab40b5a84c62eccda9e31"
server_key_version: "encryptionkey@0"
I20260806 19:43:32.537801 343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "33d451c5a9c74f1cb09aeae96770b392"
format_stamp: "Formatted at 2026-08-06 19:43:32 on dist-test-slave-77v9"
server_key: "cbf1be3db17426790ea67cee5e2c4d66"
server_key_iv: "4f58f896116ab40b5a84c62eccda9e31"
server_key_version: "encryptionkey@0"
I20260806 19:43:32.543795 343 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:43:32.548254 358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:32.549837 343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:43:32.550064 343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "33d451c5a9c74f1cb09aeae96770b392"
format_stamp: "Formatted at 2026-08-06 19:43:32 on dist-test-slave-77v9"
server_key: "cbf1be3db17426790ea67cee5e2c4d66"
server_key_iv: "4f58f896116ab40b5a84c62eccda9e31"
server_key_version: "encryptionkey@0"
I20260806 19:43:32.550294 343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:32.569274 343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:32.573788 343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:32.574105 343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:32.593232 343 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:42629
I20260806 19:43:32.593282 409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:42629 every 8 connection(s)
I20260806 19:43:32.594668 343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:43:32.595923 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 343
I20260806 19:43:32.596138 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:43:32.597005 22313 external_mini_cluster.cc:1428] Setting key e1db94179b5e0c53248c56c47406674c
I20260806 19:43:32.599505 410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 19:43:32 dist-test-slave-77v9 krb5kdc[330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:43:32.613080 410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392: Bootstrap starting.
I20260806 19:43:32.616294 410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:32.617404 410 log.cc:824] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392: Log is configured to *not* fsync() on all Append() calls
I20260806 19:43:32.620726 410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392: No bootstrap required, opened a new log
I20260806 19:43:32.620775 413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:32.599128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34786 (local address 127.21.202.126:42629)
0806 19:43:32.599912 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:32.599924 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:32.599965 (+ 41us) server_negotiation.cc:407] Connection header received
0806 19:43:32.601679 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:32.601700 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:32.602083 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:32.602414 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:32.604337 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:32.605932 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:32.607759 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:32.608496 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:32.608643 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:32.613413 (+ 4770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:32.613421 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:32.613436 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:32.613473 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:32.616465 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:32.617227 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:32.617238 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:32.617244 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:32.617427 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:32.617802 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:32.617808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:32.617811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:32.618364 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:32.618711 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:32.619103 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 19:43:32.619654 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":370,"threads_started":1}
I20260806 19:43:32.629096 410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d451c5a9c74f1cb09aeae96770b392" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42629 } }
I20260806 19:43:32.629558 410 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:32.629667 410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d451c5a9c74f1cb09aeae96770b392, State: Initialized, Role: FOLLOWER
I20260806 19:43:32.630205 410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d451c5a9c74f1cb09aeae96770b392" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42629 } }
I20260806 19:43:32.630440 410 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:43:32.630582 410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:43:32.630766 410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:32.632999 410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d451c5a9c74f1cb09aeae96770b392" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42629 } }
I20260806 19:43:32.633492 410 leader_election.cc:302] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33d451c5a9c74f1cb09aeae96770b392; no voters:
I20260806 19:43:32.634151 410 leader_election.cc:288] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:43:32.634274 415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:43:32.635464 415 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [term 1 LEADER]: Becoming Leader. State: Replica: 33d451c5a9c74f1cb09aeae96770b392, State: Running, Role: LEADER
I20260806 19:43:32.635975 415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d451c5a9c74f1cb09aeae96770b392" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42629 } }
I20260806 19:43:32.636221 410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:43:32.639075 417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33d451c5a9c74f1cb09aeae96770b392. Latest consensus state: current_term: 1 leader_uuid: "33d451c5a9c74f1cb09aeae96770b392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d451c5a9c74f1cb09aeae96770b392" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42629 } } }
I20260806 19:43:32.639511 417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [sys.catalog]: This master's current role is: LEADER
I20260806 19:43:32.640652 416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33d451c5a9c74f1cb09aeae96770b392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d451c5a9c74f1cb09aeae96770b392" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42629 } } }
I20260806 19:43:32.641050 416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392 [sys.catalog]: This master's current role is: LEADER
I20260806 19:43:32.642213 424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:43:32.647244 424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:43:32.656718 424 catalog_manager.cc:1347] Generated new cluster ID: bd5db555170444548b7ae029a32280d5
I20260806 19:43:32.656880 424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:43:32.684584 424 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:43:32.686120 424 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:43:32.694110 424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392: Generated new TSK 0
I20260806 19:43:32.695065 424 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:43:32.784883 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42629
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:43:33.145387 437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:33.145663 437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:33.145756 437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:33.153142 437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:33.153254 437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:33.153378 437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:43:33.163357 437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:33.164637 437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:33.165993 437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:33.179785 442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:33.179812 445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:33.179881 443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:33.180773 437 server_base.cc:1034] running on GCE node
I20260806 19:43:33.181442 437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:33.182399 437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:33.183620 437 hybrid_clock.cc:648] HybridClock initialized: now 1786045413183572 us; error 38 us; skew 500 ppm
I20260806 19:43:33.183972 437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:43:33 dist-test-slave-77v9 krb5kdc[330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045413, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:43:33.187398 437 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:43:33.188911 437 webserver.cc:460] Webserver started at http://127.21.202.65:46345/ using document root <none> and password file <none>
I20260806 19:43:33.189589 437 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:33.189687 437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:33.189949 437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:33.193033 437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "ceb49ef448d04a56a228fcecb64bcd46"
format_stamp: "Formatted at 2026-08-06 19:43:33 on dist-test-slave-77v9"
server_key: "8a8304a42b7dbf78f8b263b315c81df8"
server_key_iv: "3acffe4fdac7acb62e315a06639939c4"
server_key_version: "encryptionkey@0"
I20260806 19:43:33.194020 437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "ceb49ef448d04a56a228fcecb64bcd46"
format_stamp: "Formatted at 2026-08-06 19:43:33 on dist-test-slave-77v9"
server_key: "8a8304a42b7dbf78f8b263b315c81df8"
server_key_iv: "3acffe4fdac7acb62e315a06639939c4"
server_key_version: "encryptionkey@0"
I20260806 19:43:33.199936 437 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 19:43:33.203821 453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:33.205322 437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:43:33.205483 437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "ceb49ef448d04a56a228fcecb64bcd46"
format_stamp: "Formatted at 2026-08-06 19:43:33 on dist-test-slave-77v9"
server_key: "8a8304a42b7dbf78f8b263b315c81df8"
server_key_iv: "3acffe4fdac7acb62e315a06639939c4"
server_key_version: "encryptionkey@0"
I20260806 19:43:33.205698 437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:33.243888 437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:33.248232 437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:33.248495 437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:33.249645 437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:33.251061 437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:33.251139 437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:33.251231 437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:33.251276 437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:33.286331 437 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:44677
I20260806 19:43:33.286376 566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:44677 every 8 connection(s)
I20260806 19:43:33.287657 437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:43:33.291186 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 437
I20260806 19:43:33.291404 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:43:33.292320 22313 external_mini_cluster.cc:1428] Setting key a0a92e8e01579552d29849993fe237d2
Aug 06 19:43:33 dist-test-slave-77v9 krb5kdc[330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045413, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:43:33.310727 569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:33.290653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:45757 (local address 127.21.202.126:42629)
0806 19:43:33.291226 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:33.291231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:33.292438 (+ 1207us) server_negotiation.cc:407] Connection header received
0806 19:43:33.294633 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:33.294641 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:33.294761 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:33.294912 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:33.297066 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:33.297885 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:33.299658 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:33.300104 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:33.300258 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:33.304494 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:33.304500 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:33.304504 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:33.304541 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:33.307404 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:33.308203 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:33.308209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:33.308213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:33.308340 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:33.308783 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:33.308787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:33.308789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:33.309134 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:33.309318 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:33.310083 (+ 765us) server_negotiation.cc:299] Negotiation successful
0806 19:43:33.310320 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":316,"threads_started":1}
I20260806 19:43:33.311857 570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:33.291245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42629 (local address 127.21.202.65:45757)
0806 19:43:33.292226 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:33.292280 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:33.294313 (+ 2033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:33.295210 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:33.295222 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:33.296026 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:33.296798 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:33.296881 (+ 83us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:33.298083 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:33.298088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:33.299486 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:33.299499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:33.300397 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:33.300405 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:33.300676 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:33.301514 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:33.301546 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:33.304213 (+ 2667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:43:33.307565 (+ 3352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:33.307575 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:33.307594 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:33.308063 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:33.308462 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:33.308467 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:33.308471 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:33.308658 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:33.309266 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:43:33.309274 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:43:33.309648 (+ 374us) client_negotiation.cc:769] Sending connection context
0806 19:43:33.309962 (+ 314us) client_negotiation.cc:240] Negotiation successful
0806 19:43:33.310525 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":421,"threads_started":1}
I20260806 19:43:33.313582 567 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42629
I20260806 19:43:33.313931 567 heartbeater.cc:464] Registering TS with master...
I20260806 19:43:33.314844 567 heartbeater.cc:511] Master 127.21.202.126:42629 requested a full tablet report, sending...
I20260806 19:43:33.317234 375 ts_manager.cc:194] Registered new tserver with Master: ceb49ef448d04a56a228fcecb64bcd46 (127.21.202.65:44677)
I20260806 19:43:33.318703 375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:45757
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:43:33.357450 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42629
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:43:33.707767 574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:33.708058 574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:33.708158 574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:33.715637 574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:33.715757 574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:33.715893 574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:43:33.725834 574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:33.727176 574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:33.728454 574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:33.742786 579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:33.742691 580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:33.742736 582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:33.746598 574 server_base.cc:1034] running on GCE node
I20260806 19:43:33.747205 574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:33.748167 574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:33.749362 574 hybrid_clock.cc:648] HybridClock initialized: now 1786045413749316 us; error 35 us; skew 500 ppm
I20260806 19:43:33.749691 574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:43:33 dist-test-slave-77v9 krb5kdc[330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045413, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:43:33.753177 574 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:43:33.755026 574 webserver.cc:460] Webserver started at http://127.21.202.66:35649/ using document root <none> and password file <none>
I20260806 19:43:33.755795 574 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:33.755936 574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:33.756278 574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:33.759501 574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "9e8261873bf14d62ad60d0cf89bebe01"
format_stamp: "Formatted at 2026-08-06 19:43:33 on dist-test-slave-77v9"
server_key: "a222530f8a0a65dab173361f7ca1e99e"
server_key_iv: "4c18f9f0716df61bd8b5385e08f54403"
server_key_version: "encryptionkey@0"
I20260806 19:43:33.760355 574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "9e8261873bf14d62ad60d0cf89bebe01"
format_stamp: "Formatted at 2026-08-06 19:43:33 on dist-test-slave-77v9"
server_key: "a222530f8a0a65dab173361f7ca1e99e"
server_key_iv: "4c18f9f0716df61bd8b5385e08f54403"
server_key_version: "encryptionkey@0"
I20260806 19:43:33.765969 574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 19:43:33.769531 590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:33.770979 574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:43:33.771162 574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "9e8261873bf14d62ad60d0cf89bebe01"
format_stamp: "Formatted at 2026-08-06 19:43:33 on dist-test-slave-77v9"
server_key: "a222530f8a0a65dab173361f7ca1e99e"
server_key_iv: "4c18f9f0716df61bd8b5385e08f54403"
server_key_version: "encryptionkey@0"
I20260806 19:43:33.771390 574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:33.788998 574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:33.793753 574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:33.794091 574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:33.795123 574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:33.796548 574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:33.796654 574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:33.796800 574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:33.796900 574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 19:43:33.831928 574 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:44489
I20260806 19:43:33.832024 705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:44489 every 8 connection(s)
I20260806 19:43:33.833278 574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:43:33.837492 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 574
I20260806 19:43:33.837709 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:43:33.838297 22313 external_mini_cluster.cc:1428] Setting key 88087925a0204ff09b591c35568bc3b4
Aug 06 19:43:33 dist-test-slave-77v9 krb5kdc[330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045413, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:43:33.855175 708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:33.835828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:55099 (local address 127.21.202.126:42629)
0806 19:43:33.836401 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:33.836423 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:33.837352 (+ 929us) server_negotiation.cc:407] Connection header received
0806 19:43:33.839568 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:33.839574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:33.839670 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:33.839793 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:33.841584 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:33.842348 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:33.843937 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:33.844387 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:33.844527 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:33.848683 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:33.848689 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:33.848693 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:33.848730 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:33.851590 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:33.852416 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:33.852424 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:33.852428 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:33.852576 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:33.853129 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:33.853133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:33.853135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:33.853424 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:33.853596 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:33.854428 (+ 832us) server_negotiation.cc:299] Negotiation successful
0806 19:43:33.854680 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":256,"threads_started":1}
I20260806 19:43:33.856256 709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:33.836359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42629 (local address 127.21.202.66:55099)
0806 19:43:33.837199 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:33.837240 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:33.839290 (+ 2050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:33.840046 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:33.840054 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:33.840777 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:33.841402 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:33.841420 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:33.842526 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:33.842529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:33.843789 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:33.843801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:33.844641 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:33.844648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:33.844931 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:33.845630 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:33.845658 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:33.848411 (+ 2753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:43:33.851759 (+ 3348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:33.851770 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:33.851789 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:33.852268 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:33.852703 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:33.852708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:33.852713 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:33.852995 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:33.853565 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:43:33.853574 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:43:33.854005 (+ 431us) client_negotiation.cc:769] Sending connection context
0806 19:43:33.854304 (+ 299us) client_negotiation.cc:240] Negotiation successful
0806 19:43:33.854921 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":278,"threads_started":1}
I20260806 19:43:33.858049 706 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42629
I20260806 19:43:33.858409 706 heartbeater.cc:464] Registering TS with master...
I20260806 19:43:33.859387 706 heartbeater.cc:511] Master 127.21.202.126:42629 requested a full tablet report, sending...
I20260806 19:43:33.861227 375 ts_manager.cc:194] Registered new tserver with Master: 9e8261873bf14d62ad60d0cf89bebe01 (127.21.202.66:44489)
I20260806 19:43:33.862545 375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:55099
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:43:33.905447 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42629
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:43:34.254889 712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:43:34.255219 712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:43:34.255409 712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:43:34.262826 712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:43:34.262934 712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:43:34.263095 712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:43:34.273260 712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:43:34.274605 712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:43:34.275949 712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:43:34.290130 718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:34.290637 717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:43:34.290704 720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:43:34.291539 712 server_base.cc:1034] running on GCE node
I20260806 19:43:34.292191 712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:43:34.293251 712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:43:34.294490 712 hybrid_clock.cc:648] HybridClock initialized: now 1786045414294447 us; error 33 us; skew 500 ppm
I20260806 19:43:34.294823 712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:43:34 dist-test-slave-77v9 krb5kdc[330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045414, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:43:34.298607 712 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:43:34.300117 712 webserver.cc:460] Webserver started at http://127.21.202.67:36431/ using document root <none> and password file <none>
I20260806 19:43:34.300868 712 fs_manager.cc:359] Metadata directory not provided
I20260806 19:43:34.300998 712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:43:34.301327 712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:43:34.304261 712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "dcbed2bfdf144c80afd70f7ee35abca2"
format_stamp: "Formatted at 2026-08-06 19:43:34 on dist-test-slave-77v9"
server_key: "042eb5d15f4c516581f68ea16f8f353d"
server_key_iv: "7c628a68b1e1489dfa9e7aabefe0d504"
server_key_version: "encryptionkey@0"
I20260806 19:43:34.305094 712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "dcbed2bfdf144c80afd70f7ee35abca2"
format_stamp: "Formatted at 2026-08-06 19:43:34 on dist-test-slave-77v9"
server_key: "042eb5d15f4c516581f68ea16f8f353d"
server_key_iv: "7c628a68b1e1489dfa9e7aabefe0d504"
server_key_version: "encryptionkey@0"
I20260806 19:43:34.310467 712 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 19:43:34.314050 728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:34.315426 712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:43:34.315555 712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "dcbed2bfdf144c80afd70f7ee35abca2"
format_stamp: "Formatted at 2026-08-06 19:43:34 on dist-test-slave-77v9"
server_key: "042eb5d15f4c516581f68ea16f8f353d"
server_key_iv: "7c628a68b1e1489dfa9e7aabefe0d504"
server_key_version: "encryptionkey@0"
I20260806 19:43:34.315732 712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:43:34.321851 567 heartbeater.cc:503] Master 127.21.202.126:42629 was elected leader, sending a full tablet report...
I20260806 19:43:34.354615 712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:43:34.359218 712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:43:34.359493 712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:43:34.360466 712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:43:34.361827 712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:43:34.361908 712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:34.362003 712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:43:34.362049 712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:43:34.398270 712 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:35021
I20260806 19:43:34.398345 846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:35021 every 8 connection(s)
I20260806 19:43:34.399722 712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:43:34.405645 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 712
I20260806 19:43:34.405838 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:43:34.406299 22313 external_mini_cluster.cc:1428] Setting key 2e049ffb75667b4fabdca48b45a51f17
Aug 06 19:43:34 dist-test-slave-77v9 krb5kdc[330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045414, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:43:34.420791 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.402347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:35357 (local address 127.21.202.126:42629)
0806 19:43:34.402891 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:34.402896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:34.403703 (+ 807us) server_negotiation.cc:407] Connection header received
0806 19:43:34.405693 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:34.405699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:34.405823 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:34.405942 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:34.407760 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.408851 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.410648 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.411108 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.411208 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.414635 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:34.414641 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:34.414644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:34.414682 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:34.417658 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:34.418371 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:34.418375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:34.418378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:34.418513 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:34.418932 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:34.418935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:34.418938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:34.419216 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:34.419359 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:34.420110 (+ 751us) server_negotiation.cc:299] Negotiation successful
0806 19:43:34.420348 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":270,"threads_started":1}
I20260806 19:43:34.421916 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.402796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42629 (local address 127.21.202.67:35357)
0806 19:43:34.403548 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:34.403585 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:34.405402 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:34.406183 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:34.406192 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:34.406926 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:43:34.407554 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:34.407572 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.409035 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.409040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:34.410484 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:34.410497 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.411316 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.411321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:34.411521 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.412213 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:43:34.412242 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:43:34.414425 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:43:34.417814 (+ 3389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:34.417820 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:34.417832 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:34.418259 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:34.418606 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:43:34.418609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:43:34.418611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:43:34.418782 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:43:34.419330 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:43:34.419336 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:43:34.419671 (+ 335us) client_negotiation.cc:769] Sending connection context
0806 19:43:34.419988 (+ 317us) client_negotiation.cc:240] Negotiation successful
0806 19:43:34.420507 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":316,"threads_started":1}
I20260806 19:43:34.423277 847 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42629
I20260806 19:43:34.423610 847 heartbeater.cc:464] Registering TS with master...
I20260806 19:43:34.424443 847 heartbeater.cc:511] Master 127.21.202.126:42629 requested a full tablet report, sending...
I20260806 19:43:34.426028 375 ts_manager.cc:194] Registered new tserver with Master: dcbed2bfdf144c80afd70f7ee35abca2 (127.21.202.67:35021)
I20260806 19:43:34.427019 375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:35357
I20260806 19:43:34.434331 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:43:34.453457 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.439886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34794 (local address 127.21.202.126:42629)
0806 19:43:34.440125 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:34.440134 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:34.440602 (+ 468us) server_negotiation.cc:407] Connection header received
0806 19:43:34.440746 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:34.440750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:34.440881 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:34.440992 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:34.442478 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.443286 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.444603 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.445117 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.445221 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.448075 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:34.448081 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:34.448084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:34.448117 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:34.450905 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:34.451560 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:34.451564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:34.451566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:34.451685 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:34.452105 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:34.452109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:34.452111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:34.452433 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:34.452615 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:34.453037 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 19:43:34.453208 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 19:43:34.458122 375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:43:34.465643 375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:43:34.497145 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.480211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54500 (local address 127.21.202.66:44489)
0806 19:43:34.480899 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:34.480912 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:34.481499 (+ 587us) server_negotiation.cc:407] Connection header received
0806 19:43:34.484057 (+ 2558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:34.484071 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:34.484412 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:34.484654 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:43:34.487227 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.489390 (+ 2163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.493782 (+ 4392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.495325 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.495468 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.495699 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:34.496567 (+ 868us) server_negotiation.cc:299] Negotiation successful
0806 19:43:34.496763 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":304,"threads_started":1}
I20260806 19:43:34.497555 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.479944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:44489 (local address 127.0.0.1:54500)
0806 19:43:34.481214 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:34.481233 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:34.481356 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:34.485036 (+ 3680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:34.485047 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:34.485091 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:43:34.486950 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:34.486969 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.489620 (+ 2651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.489629 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:34.493599 (+ 3970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:34.493619 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.495859 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.495871 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:34.496141 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.496308 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 19:43:34.496463 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 19:43:34.496671 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"mutex_wait_us":79,"thread_start_us":984,"threads_started":1}
I20260806 19:43:34.500231 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.479048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47074 (local address 127.21.202.65:44677)
0806 19:43:34.491781 (+ 12733us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:34.491792 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:34.491827 (+ 35us) server_negotiation.cc:407] Connection header received
0806 19:43:34.491933 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:34.491943 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:34.492235 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:34.493190 (+ 955us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:43:34.493445 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.494537 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.497337 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.498873 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.499143 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.499393 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:34.499711 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 19:43:34.499881 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12533,"thread_start_us":12798,"threads_started":1}
I20260806 19:43:34.500293 860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.478858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:44677 (local address 127.0.0.1:47074)
0806 19:43:34.479706 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:34.479769 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:34.479981 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:34.492485 (+ 12504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:34.492491 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:34.492517 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:43:34.492986 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:34.493000 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.494673 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.494681 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:34.496782 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:34.496795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.499273 (+ 2478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.499285 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:34.499520 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.499544 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 19:43:34.499664 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 19:43:34.499806 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":5,"thread_start_us":306,"threads_started":1}
I20260806 19:43:34.500921 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.481288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:35021 (local address 127.0.0.1:32958)
0806 19:43:34.491518 (+ 10230us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:34.491544 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:34.491700 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:34.493161 (+ 1461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:34.493165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:34.493191 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:43:34.493627 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:34.493638 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.495622 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.495627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:34.498142 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:34.498155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.500099 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.500110 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:34.500288 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.500320 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 19:43:34.500408 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 19:43:34.500529 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10042,"mutex_wait_us":43,"thread_start_us":9828,"threads_started":1}
I20260806 19:43:34.501775 865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.481754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32958 (local address 127.21.202.67:35021)
0806 19:43:34.492644 (+ 10890us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:34.492650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:34.492675 (+ 25us) server_negotiation.cc:407] Connection header received
0806 19:43:34.492747 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:34.492752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:34.492980 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:34.493174 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:43:34.493861 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.495468 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.498294 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.499575 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.499707 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.499934 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:34.501257 (+ 1323us) server_negotiation.cc:299] Negotiation successful
0806 19:43:34.501426 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10740,"thread_start_us":10568,"threads_started":1}
I20260806 19:43:34.502213 639 tablet_service.cc:1467] Processing CreateTablet for tablet f37f2d50fe9346bdaec317fd7fc35411 (DEFAULT_TABLE table=test-table [id=d01c993b4dca43738db71cdf5bb8cfe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:43:34.503489 639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f37f2d50fe9346bdaec317fd7fc35411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:34.507545 502 tablet_service.cc:1467] Processing CreateTablet for tablet f37f2d50fe9346bdaec317fd7fc35411 (DEFAULT_TABLE table=test-table [id=d01c993b4dca43738db71cdf5bb8cfe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:43:34.508172 779 tablet_service.cc:1467] Processing CreateTablet for tablet f37f2d50fe9346bdaec317fd7fc35411 (DEFAULT_TABLE table=test-table [id=d01c993b4dca43738db71cdf5bb8cfe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:43:34.509171 502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f37f2d50fe9346bdaec317fd7fc35411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:34.509578 779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f37f2d50fe9346bdaec317fd7fc35411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:43:34.520360 866 tablet_bootstrap.cc:492] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01: Bootstrap starting.
I20260806 19:43:34.523622 866 tablet_bootstrap.cc:654] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:34.524286 868 tablet_bootstrap.cc:492] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2: Bootstrap starting.
I20260806 19:43:34.525125 867 tablet_bootstrap.cc:492] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46: Bootstrap starting.
I20260806 19:43:34.525115 866 log.cc:824] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01: Log is configured to *not* fsync() on all Append() calls
I20260806 19:43:34.528016 868 tablet_bootstrap.cc:654] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:34.529340 868 log.cc:824] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2: Log is configured to *not* fsync() on all Append() calls
I20260806 19:43:34.529505 867 tablet_bootstrap.cc:654] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46: Neither blocks nor log segments found. Creating new log.
I20260806 19:43:34.530675 867 log.cc:824] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46: Log is configured to *not* fsync() on all Append() calls
I20260806 19:43:34.531900 866 tablet_bootstrap.cc:492] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01: No bootstrap required, opened a new log
I20260806 19:43:34.532167 866 ts_tablet_manager.cc:1397] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260806 19:43:34.533149 868 tablet_bootstrap.cc:492] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2: No bootstrap required, opened a new log
I20260806 19:43:34.533401 868 ts_tablet_manager.cc:1397] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 19:43:34.534117 867 tablet_bootstrap.cc:492] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46: No bootstrap required, opened a new log
I20260806 19:43:34.534356 867 ts_tablet_manager.cc:1397] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260806 19:43:34.541517 868 raft_consensus.cc:348] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb49ef448d04a56a228fcecb64bcd46" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 44677 } } peers { permanent_uuid: "9e8261873bf14d62ad60d0cf89bebe01" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 44489 } } peers { permanent_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35021 } }
I20260806 19:43:34.542124 868 raft_consensus.cc:374] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:34.542237 868 raft_consensus.cc:729] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcbed2bfdf144c80afd70f7ee35abca2, State: Initialized, Role: FOLLOWER
I20260806 19:43:34.542843 868 consensus_queue.cc:260] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb49ef448d04a56a228fcecb64bcd46" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 44677 } } peers { permanent_uuid: "9e8261873bf14d62ad60d0cf89bebe01" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 44489 } } peers { permanent_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35021 } }
I20260806 19:43:34.543085 866 raft_consensus.cc:348] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb49ef448d04a56a228fcecb64bcd46" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 44677 } } peers { permanent_uuid: "9e8261873bf14d62ad60d0cf89bebe01" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 44489 } } peers { permanent_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35021 } }
I20260806 19:43:34.543593 866 raft_consensus.cc:374] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:34.543982 847 heartbeater.cc:503] Master 127.21.202.126:42629 was elected leader, sending a full tablet report...
I20260806 19:43:34.544518 868 ts_tablet_manager.cc:1428] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260806 19:43:34.543730 866 raft_consensus.cc:729] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e8261873bf14d62ad60d0cf89bebe01, State: Initialized, Role: FOLLOWER
I20260806 19:43:34.545058 867 raft_consensus.cc:348] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb49ef448d04a56a228fcecb64bcd46" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 44677 } } peers { permanent_uuid: "9e8261873bf14d62ad60d0cf89bebe01" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 44489 } } peers { permanent_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35021 } }
I20260806 19:43:34.545274 866 consensus_queue.cc:260] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb49ef448d04a56a228fcecb64bcd46" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 44677 } } peers { permanent_uuid: "9e8261873bf14d62ad60d0cf89bebe01" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 44489 } } peers { permanent_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35021 } }
I20260806 19:43:34.545504 867 raft_consensus.cc:374] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:43:34.545614 867 raft_consensus.cc:729] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb49ef448d04a56a228fcecb64bcd46, State: Initialized, Role: FOLLOWER
I20260806 19:43:34.546423 867 consensus_queue.cc:260] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb49ef448d04a56a228fcecb64bcd46" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 44677 } } peers { permanent_uuid: "9e8261873bf14d62ad60d0cf89bebe01" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 44489 } } peers { permanent_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35021 } }
I20260806 19:43:34.546777 706 heartbeater.cc:503] Master 127.21.202.126:42629 was elected leader, sending a full tablet report...
I20260806 19:43:34.547502 866 ts_tablet_manager.cc:1428] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260806 19:43:34.549463 867 ts_tablet_manager.cc:1428] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260806 19:43:34.585412 707 tablet.cc:2369] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:43:34.651547 848 tablet.cc:2369] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:43:34.736932 872 raft_consensus.cc:482] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:43:34.737288 872 raft_consensus.cc:504] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb49ef448d04a56a228fcecb64bcd46" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 44677 } } peers { permanent_uuid: "9e8261873bf14d62ad60d0cf89bebe01" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 44489 } } peers { permanent_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35021 } }
I20260806 19:43:34.739243 872 leader_election.cc:288] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb49ef448d04a56a228fcecb64bcd46 (127.21.202.65:44677), 9e8261873bf14d62ad60d0cf89bebe01 (127.21.202.66:44489)
I20260806 19:43:34.745980 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.739621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:44677 (local address 127.21.202.67:44087)
0806 19:43:34.739855 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:34.739876 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:34.740012 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:34.740453 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:34.740459 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:34.740491 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:43:34.740737 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:34.740751 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.742147 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.742151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:34.744068 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:34.744082 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.745412 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.745416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:34.745555 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.745580 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 19:43:34.745656 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 19:43:34.745750 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 19:43:34.746104 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.739704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:44087 (local address 127.21.202.65:44677)
0806 19:43:34.739900 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:34.739907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:34.739942 (+ 35us) server_negotiation.cc:407] Connection header received
0806 19:43:34.740168 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:34.740173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:34.740246 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:34.740354 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:43:34.740971 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.742003 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.744228 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.745254 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.745418 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.745510 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:34.745710 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 19:43:34.745851 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 19:43:34.747124 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.740094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:44489 (local address 127.21.202.67:40203)
0806 19:43:34.740916 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:34.740931 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 19:43:34.741026 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:43:34.741328 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:43:34.741331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:43:34.741350 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:43:34.741639 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:34.741646 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.742756 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.742761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:34.744636 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:43:34.744646 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.746473 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.746480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:43:34.746649 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.746686 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:43:34.746783 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 19:43:34.746872 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":595,"threads_started":1}
I20260806 19:43:34.747228 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.740645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:40203 (local address 127.21.202.66:44489)
0806 19:43:34.740898 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:34.740903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:34.740983 (+ 80us) server_negotiation.cc:407] Connection header received
0806 19:43:34.741119 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:34.741123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:34.741194 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:34.741294 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:43:34.741784 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.742591 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.744782 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.746054 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.746242 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.746316 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:34.746830 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 19:43:34.746976 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 19:43:34.747315 522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f37f2d50fe9346bdaec317fd7fc35411" candidate_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb49ef448d04a56a228fcecb64bcd46" is_pre_election: true
I20260806 19:43:34.747979 522 raft_consensus.cc:2454] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcbed2bfdf144c80afd70f7ee35abca2 in term 0.
I20260806 19:43:34.748107 659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f37f2d50fe9346bdaec317fd7fc35411" candidate_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e8261873bf14d62ad60d0cf89bebe01" is_pre_election: true
I20260806 19:43:34.748617 659 raft_consensus.cc:2454] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcbed2bfdf144c80afd70f7ee35abca2 in term 0.
I20260806 19:43:34.748648 729 leader_election.cc:302] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb49ef448d04a56a228fcecb64bcd46, dcbed2bfdf144c80afd70f7ee35abca2; no voters:
I20260806 19:43:34.749146 872 raft_consensus.cc:2781] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:43:34.749295 872 raft_consensus.cc:482] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:43:34.749409 872 raft_consensus.cc:3037] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:34.751660 872 raft_consensus.cc:504] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb49ef448d04a56a228fcecb64bcd46" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 44677 } } peers { permanent_uuid: "9e8261873bf14d62ad60d0cf89bebe01" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 44489 } } peers { permanent_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35021 } }
I20260806 19:43:34.752239 872 leader_election.cc:288] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [CANDIDATE]: Term 1 election: Requested vote from peers ceb49ef448d04a56a228fcecb64bcd46 (127.21.202.65:44677), 9e8261873bf14d62ad60d0cf89bebe01 (127.21.202.66:44489)
I20260806 19:43:34.752878 522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f37f2d50fe9346bdaec317fd7fc35411" candidate_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb49ef448d04a56a228fcecb64bcd46"
I20260806 19:43:34.752975 659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f37f2d50fe9346bdaec317fd7fc35411" candidate_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e8261873bf14d62ad60d0cf89bebe01"
I20260806 19:43:34.753123 522 raft_consensus.cc:3037] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:34.753161 659 raft_consensus.cc:3037] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:43:34.756438 659 raft_consensus.cc:2454] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcbed2bfdf144c80afd70f7ee35abca2 in term 1.
I20260806 19:43:34.756446 522 raft_consensus.cc:2454] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcbed2bfdf144c80afd70f7ee35abca2 in term 1.
I20260806 19:43:34.756906 729 leader_election.cc:302] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ceb49ef448d04a56a228fcecb64bcd46, dcbed2bfdf144c80afd70f7ee35abca2; no voters:
I20260806 19:43:34.757228 872 raft_consensus.cc:2781] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:43:34.757732 872 raft_consensus.cc:686] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [term 1 LEADER]: Becoming Leader. State: Replica: dcbed2bfdf144c80afd70f7ee35abca2, State: Running, Role: LEADER
I20260806 19:43:34.758304 872 consensus_queue.cc:237] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb49ef448d04a56a228fcecb64bcd46" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 44677 } } peers { permanent_uuid: "9e8261873bf14d62ad60d0cf89bebe01" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 44489 } } peers { permanent_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35021 } }
I20260806 19:43:34.763051 375 catalog_manager.cc:5495] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 reported cstate change: term changed from 0 to 1, leader changed from <none> to dcbed2bfdf144c80afd70f7ee35abca2 (127.21.202.67). New cstate: current_term: 1 leader_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb49ef448d04a56a228fcecb64bcd46" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 44677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e8261873bf14d62ad60d0cf89bebe01" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 44489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcbed2bfdf144c80afd70f7ee35abca2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35021 } health_report { overall_health: HEALTHY } } }
I20260806 19:43:34.787025 865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.780882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32972 (local address 127.21.202.67:35021)
0806 19:43:34.781121 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:34.781128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:34.781236 (+ 108us) server_negotiation.cc:407] Connection header received
0806 19:43:34.781495 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:34.781500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:34.781593 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:34.781746 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:43:34.782125 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.782890 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.784331 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.785075 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.785223 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.785865 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:43:34.786395 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:43:34.786574 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:34.786634 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 19:43:34.786726 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 19:43:34.790473 568 tablet.cc:2369] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:43:34.795828 522 raft_consensus.cc:1261] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46 [term 1 FOLLOWER]: Refusing update from remote peer dcbed2bfdf144c80afd70f7ee35abca2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:43:34.796042 659 raft_consensus.cc:1261] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01 [term 1 FOLLOWER]: Refusing update from remote peer dcbed2bfdf144c80afd70f7ee35abca2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:43:34.796909 876 consensus_queue.cc:1035] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb49ef448d04a56a228fcecb64bcd46" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 44677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:43:34.797312 872 consensus_queue.cc:1035] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e8261873bf14d62ad60d0cf89bebe01" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:43:34.826615 374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:43:34.826910 374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:43:34.830830 374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33d451c5a9c74f1cb09aeae96770b392: Sending DeleteTablet for 3 replicas of tablet f37f2d50fe9346bdaec317fd7fc35411
I20260806 19:43:34.832554 779 tablet_service.cc:1514] Processing DeleteTablet for tablet f37f2d50fe9346bdaec317fd7fc35411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:43:34 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:32958
I20260806 19:43:34.832919 502 tablet_service.cc:1514] Processing DeleteTablet for tablet f37f2d50fe9346bdaec317fd7fc35411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:43:34 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:47074
I20260806 19:43:34.833995 639 tablet_service.cc:1514] Processing DeleteTablet for tablet f37f2d50fe9346bdaec317fd7fc35411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:43:34 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:54500
I20260806 19:43:34.840484 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 437
I20260806 19:43:34.853652 890 tablet_replica.cc:330] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2: stopping tablet replica
I20260806 19:43:34.854207 890 raft_consensus.cc:2229] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:43:34.854938 890 raft_consensus.cc:2258] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:34.859090 890 ts_tablet_manager.cc:1905] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 19:43:34 dist-test-slave-77v9 krb5kdc[330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:43:34.869187 892 tablet_replica.cc:330] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01: stopping tablet replica
I20260806 19:43:34.869750 892 raft_consensus.cc:2229] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:43:34.871418 891 tablet_replica.cc:330] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46: stopping tablet replica
I20260806 19:43:34.874251 891 raft_consensus.cc:2229] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:43:34.874823 892 raft_consensus.cc:2258] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:34.875409 890 ts_tablet_manager.cc:1918] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:43:34.875674 890 log.cc:1196] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/f37f2d50fe9346bdaec317fd7fc35411
I20260806 19:43:34.876350 890 ts_tablet_manager.cc:1939] T f37f2d50fe9346bdaec317fd7fc35411 P dcbed2bfdf144c80afd70f7ee35abca2: Deleting consensus metadata
I20260806 19:43:34.876850 891 raft_consensus.cc:2258] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:43:34.877596 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.841938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47090 (local address 127.21.202.65:44677)
0806 19:43:34.842148 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:34.842156 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:34.845436 (+ 3280us) server_negotiation.cc:407] Connection header received
0806 19:43:34.845787 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:34.845796 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:34.845927 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:34.846034 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:34.855450 (+ 9416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.856326 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.858253 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:34.858916 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:34.859013 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:34.870412 (+ 11399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:34.870431 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:34.870449 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:34.870496 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:34.873703 (+ 3207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:34.874716 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:34.874725 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:34.874728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:34.874925 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:34.875438 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:34.875443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:34.875446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:34.876040 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:34.876331 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:34.876978 (+ 647us) server_negotiation.cc:299] Negotiation successful
0806 19:43:34.877222 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 19:43:34.878459 359 catalog_manager.cc:4841] TS dcbed2bfdf144c80afd70f7ee35abca2 (127.21.202.67:35021): tablet f37f2d50fe9346bdaec317fd7fc35411 (table test-table [id=d01c993b4dca43738db71cdf5bb8cfe1]) successfully deleted
I20260806 19:43:34.879240 892 ts_tablet_manager.cc:1905] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:34.879513 562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:34.886551 892 ts_tablet_manager.cc:1918] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:43:34.886727 892 log.cc:1196] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/f37f2d50fe9346bdaec317fd7fc35411
I20260806 19:43:34.887310 892 ts_tablet_manager.cc:1939] T f37f2d50fe9346bdaec317fd7fc35411 P 9e8261873bf14d62ad60d0cf89bebe01: Deleting consensus metadata
I20260806 19:43:34.889417 359 catalog_manager.cc:4841] TS 9e8261873bf14d62ad60d0cf89bebe01 (127.21.202.66:44489): tablet f37f2d50fe9346bdaec317fd7fc35411 (table test-table [id=d01c993b4dca43738db71cdf5bb8cfe1]) successfully deleted
I20260806 19:43:34.879832 891 ts_tablet_manager.cc:1905] T f37f2d50fe9346bdaec317fd7fc35411 P ceb49ef448d04a56a228fcecb64bcd46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:43:34.979079 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 437
I20260806 19:43:34.998215 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 574
W20260806 19:43:34.998440 359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.202.65:44677 (error 108)
I20260806 19:43:35.000488 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.999949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:44677 (local address 127.0.0.1:47092)
0806 19:43:35.000189 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:35.000309 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:44677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260806 19:43:35.001132 359 catalog_manager.cc:4584] TS ceb49ef448d04a56a228fcecb64bcd46 (127.21.202.65:44677): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f37f2d50fe9346bdaec317fd7fc35411: Network error: Client connection negotiation failed: client connection to 127.21.202.65:44677: connect: Connection refused (error 111)
Aug 06 19:43:35 dist-test-slave-77v9 krb5kdc[330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:43:35.014827 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:34.999149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54508 (local address 127.21.202.66:44489)
0806 19:43:34.999703 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:34.999710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:34.999731 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:43:34.999807 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:34.999811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:34.999874 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:35.000013 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:35.001793 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:35.002571 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:35.003809 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:35.004347 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:35.004515 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:35.008666 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:35.008681 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:35.008697 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:35.008733 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:35.011979 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:35.012660 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:35.012669 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:35.012673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:35.012994 (+ 321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:35.013423 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:35.013426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:35.013428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:35.013791 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:35.014104 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:35.014345 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 19:43:35.014530 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410}
I20260806 19:43:35.015563 701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:35.054879 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:35.054293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:44677 (local address 127.0.0.1:47102)
0806 19:43:35.054524 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:35.054657 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:44677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 19:43:35.056084 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:35.055670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:44677 (local address 127.0.0.1:47110)
0806 19:43:35.055832 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:35.055935 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:44677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 19:43:35.110297 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 574
I20260806 19:43:35.116832 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:35.116227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:44677 (local address 127.0.0.1:47114)
0806 19:43:35.116481 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:35.116593 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:44677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 19:43:35.118520 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:35.117861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:44677 (local address 127.0.0.1:47126)
0806 19:43:35.118231 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:35.118342 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:44677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 19:43:35.131462 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 712
Aug 06 19:43:35 dist-test-slave-77v9 krb5kdc[330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:43:35.151044 865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:35.132357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.21.202.67:35021)
0806 19:43:35.132540 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 19:43:35.132544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:43:35.132583 (+ 39us) server_negotiation.cc:407] Connection header received
0806 19:43:35.132799 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:43:35.132803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:43:35.132948 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:43:35.133033 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:43:35.134873 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:35.135758 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:35.137851 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:43:35.138306 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:43:35.138514 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:43:35.144308 (+ 5794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:43:35.144323 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:43:35.144339 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:43:35.144394 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:43:35.147950 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:35.148727 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:35.148735 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:35.148737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:35.148970 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:43:35.149384 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:43:35.149390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:43:35.149393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:43:35.149782 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:43:35.150128 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0806 19:43:35.150421 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 19:43:35.150656 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 19:43:35.151821 842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:35.199630 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:35.199075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:44677 (local address 127.0.0.1:47130)
0806 19:43:35.199321 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:35.199446 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:44677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 19:43:35.200793 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:35.200355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:44677 (local address 127.0.0.1:47140)
0806 19:43:35.200558 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:35.200650 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:44677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 19:43:35.281143 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 712
I20260806 19:43:35.303820 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 343
I20260806 19:43:35.305017 405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:43:35.414633 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 343
I20260806 19:43:35.414777 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:43:35.413854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:44677 (local address 127.0.0.1:47146)
0806 19:43:35.414093 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 19:43:35.414218 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:44677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
2026-08-06T19:43:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4741 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:43:35 dist-test-slave-77v9 krb5kdc[901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:43:35 dist-test-slave-77v9 krb5kdc[901](info): set up 2 sockets
Aug 06 19:43:35 dist-test-slave-77v9 krb5kdc[901](info): commencing operation
krb5kdc: starting...
W20260806 19:43:36.554201 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:43:36 dist-test-slave-77v9 krb5kdc[901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:43:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:43:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/rangeradmin_127.21.202.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/rangeradmin_127.21.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/rangerlookup_127.21.202.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/rangerlookup_127.21.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/HTTP_127.21.202.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/HTTP_127.21.202.84@KRBTEST.COM.keytab.
I20260806 19:43:36.976025 22313 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskha9koQ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/postgres -l logfile start
2026-08-06 19:43:39.714 UTC [932] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 19:43:39.714 UTC [932] LOG: listening on IPv4 address "127.21.202.84", port 43833
2026-08-06 19:43:39.718 UTC [932] LOG: listening on Unix socket "/tmp/.s.PGSQL.43833"
2026-08-06 19:43:39.742 UTC [935] LOG: database system was shut down at 2026-08-06 19:43:38 UTC
2026-08-06 19:43:39.747 UTC [932] LOG: database system is ready to accept connections
I20260806 19:43:40.753566 22313 mini_postgres.cc:96] Postgres bound to 43833
2026-08-06 19:43:40.779 UTC [943] FATAL: database "slave" does not exist
127.21.202.84:43833 - accepting connections
I20260806 19:43:40.780652 22313 mini_ranger.cc:161] Starting Ranger...
I20260806 19:43:40.821489 22313 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 19:43:41.359725 22313 mini_ranger.cc:87] Created ranger Postgres database
I20260806 19:43:41.359938 22313 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-admin
2026-08-06 19:43:41,750 [I] DB FLAVOR :POSTGRES
2026-08-06 19:43:41,751 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 19:43:41,751 [I] Checking connection..
2026-08-06 19:43:41,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 19:43:42,016 [I] Checking connection passed.
2026-08-06 19:43:42,016 [I] --------- Verifying version history table ---------
2026-08-06 19:43:42,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 19:43:42,286 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 19:43:42,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 19:43:42,539 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 19:43:42,540 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 19:43:42,540 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 19:43:42.804 UTC [1008] WARNING: there is no transaction in progress
2026-08-06 19:43:42,811 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 19:43:42,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 19:43:43,058 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 19:43:43,059 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 19:43:43,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 19:43:43,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 19:43:43,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-77v9.c.gcp-upstream.internal','N') ;"
2026-08-06 19:43:43,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 19:43:44,067 [I] Table x_portal_user does not exist in database ranger
2026-08-06 19:43:44,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 19:43:44,319 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 19:43:44,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 19:43:44,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 19:43:44,848 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 19:43:44,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 19:43:45.868 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:45.888 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:45.906 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:45.927 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:45.945 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:45.966 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:46.027 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:46.034 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:46.050 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:46.061 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:46.072 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:46.087 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:46.098 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:46.105 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:46.113 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:46.120 UTC [1121] WARNING: there is no transaction in progress
2026-08-06 19:43:46,865 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 19:43:46,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 19:43:47,143 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 19:43:47,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 19:43:47,392 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 19:43:47,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 19:43:47,633 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 19:43:47,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 19:43:47,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 19:43:48,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-77v9.c.gcp-upstream.internal';"
2026-08-06 19:43:48,340 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 19:43:48,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 19:43:48,600 [I] DB_PATCHES have already been applied
I20260806 19:43:48.606680 22313 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-admin:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskha9koQ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 19:43:48.606801 22313 mini_ranger.cc:193] Using host: 127.21.202.84
I20260806 19:43:48.628901 22313 mini_ranger.cc:227] Ranger admin URL: http://127.21.202.84:45053
Aug 06, 2026 7:43:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 7:43:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 7:43:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 7:43:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 7:43:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 7:43:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 7:43:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 19:43:49.829906 22313 mini_ranger.cc:160] Time spent starting Ranger: real 9.049s user 0.004s sys 0.061s
Aug 06, 2026 7:44:11 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 7:44:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 06, 2026 7:44:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 7:44:11 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 7:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 7:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 7:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 7:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 7:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 7:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 7:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 7:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 7:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 7:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 7:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 7:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 7:44:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 7:44:12 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 19:44:12.864048 22313 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/rangerkms_127.21.202.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/rangerkms_127.21.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.202.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.202.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.202.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/HTTP_127.21.202.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.202.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/HTTP_127.21.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 19:44:13 dist-test-slave-77v9 krb5kdc[901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045453, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 19:44:13.074880 22313 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 19:44:13.102941 22313 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 19:44:14.177268 22313 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 19:44:14.177412 22313 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-kms
I20260806 19:44:14.177578 22313 mini_ranger_kms.cc:227] Using postgres at 127.21.202.84:43833
2026-08-06 19:44:15,162 [I] DB FLAVOR :POSTGRES
2026-08-06 19:44:15,162 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 19:44:15,162 [I] Checking connection
2026-08-06 19:44:15,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 19:44:15,426 [I] connection success
2026-08-06 19:44:15,426 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 19:44:15,427 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 19:44:15,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 19:44:15,717 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 19:44:15,718 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 19:44:15,718 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 19:44:15,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:43833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 19:44:16,011 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 19:44:16.550694 22313 mini_ranger_kms.cc:310] Created kms service
I20260806 19:44:16.781388 22313 mini_ranger_kms.cc:326] Created kudu user
I20260806 19:44:16.832058 22313 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 19:44:16 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045453, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:44:17.277742 22313 mini_ranger_kms.cc:384] Added ranger policy
I20260806 19:44:17.279177 22313 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-kms:/tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 19:44:17.279258 22313 mini_ranger_kms.cc:240] Using host: 127.21.202.84
I20260806 19:44:17.308686 22313 mini_ranger_kms.cc:276] Ranger KMS PID: 1308
I20260806 19:44:17.308936 22313 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.202.84:36943
Aug 06, 2026 7:44:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 7:44:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 7:44:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 7:44:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 7:44:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 7:44:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 19:44:19 dist-test-slave-77v9 krb5kdc[901](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786045459, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.202.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:44:19.923718 22313 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.849s user 0.142s sys 0.494s
I20260806 19:44:19.924144 22313 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 19:44:19.924402 22313 mini_ranger_kms.cc:401] 127.21.202.84:36943/kms/v1/keys
Aug 06 19:44:20 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045459, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.202.84@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
Aug 06 19:44:20 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045459, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.202.84@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
Aug 06 19:44:23 dist-test-slave-77v9 krb5kdc[901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:44:23.270135 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:33131
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:33131
--ranger_config_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.202.84:36943/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:44:23.643680 1361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:44:23.643996 1361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:44:23.644090 1361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:44:23.651585 1361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:44:23.651765 1361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:44:23.652045 1361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:44:23.652177 1361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:44:23.652400 1361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:44:23.663905 1361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.202.84:36943/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:33131
--ranger_config_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:33131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:44:23.665408 1361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:44:23.666862 1361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:44:23.680415 1366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:44:23.680363 1367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:44:23.680413 1369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:44:23.686362 1361 server_base.cc:1034] running on GCE node
I20260806 19:44:23.687304 1361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:44:23.689008 1361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:44:23.690254 1361 hybrid_clock.cc:648] HybridClock initialized: now 1786045463690215 us; error 59 us; skew 500 ppm
I20260806 19:44:23.690665 1361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:44:23 dist-test-slave-77v9 krb5kdc[901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045463, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:44:23.694352 1361 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:44:23.696143 1361 webserver.cc:460] Webserver started at http://127.21.202.126:36533/ using document root <none> and password file <none>
I20260806 19:44:23.696976 1361 fs_manager.cc:359] Metadata directory not provided
I20260806 19:44:23.697122 1361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:44:23.697468 1361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:44:23 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045463, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:44:23.761785 1361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "e010731dbb5843a3928e5b11e650f9c8"
format_stamp: "Formatted at 2026-08-06 19:44:23 on dist-test-slave-77v9"
server_key: "ccb6968d1179b723ebf6e7798a23292a"
server_key_iv: "d3f330cfb4c619ddce4cb68c319c04da"
server_key_version: "kuduclusterkey@0"
I20260806 19:44:23.762701 1361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "e010731dbb5843a3928e5b11e650f9c8"
format_stamp: "Formatted at 2026-08-06 19:44:23 on dist-test-slave-77v9"
server_key: "ccb6968d1179b723ebf6e7798a23292a"
server_key_iv: "d3f330cfb4c619ddce4cb68c319c04da"
server_key_version: "kuduclusterkey@0"
I20260806 19:44:23.768954 1361 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 19:44:23.793527 1376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:44:23.795300 1361 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.003s
I20260806 19:44:23.795472 1361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "e010731dbb5843a3928e5b11e650f9c8"
format_stamp: "Formatted at 2026-08-06 19:44:23 on dist-test-slave-77v9"
server_key: "ccb6968d1179b723ebf6e7798a23292a"
server_key_iv: "d3f330cfb4c619ddce4cb68c319c04da"
server_key_version: "kuduclusterkey@0"
I20260806 19:44:23.795676 1361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:44:23.814466 1361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:44:23.818967 1361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:44:23.819235 1361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:44:23.839432 1361 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:33131
I20260806 19:44:23.839468 1430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:33131 every 8 connection(s)
I20260806 19:44:23.840912 1361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:44:23.845046 1431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:44:23.848955 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 1361
I20260806 19:44:23.849195 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:44:23.850157 22313 external_mini_cluster.cc:1428] Setting key e69cbca73b539d09c1dccd53a0090300
I20260806 19:44:23.859134 1431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8: Bootstrap starting.
I20260806 19:44:23.862605 1431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8: Neither blocks nor log segments found. Creating new log.
I20260806 19:44:23.863663 1431 log.cc:824] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8: Log is configured to *not* fsync() on all Append() calls
Aug 06 19:44:23 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:44:23.867203 1431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8: No bootstrap required, opened a new log
I20260806 19:44:23.874187 1434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:23.852645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39686 (local address 127.21.202.126:33131)
0806 19:44:23.853624 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:23.853636 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:23.853680 (+ 44us) server_negotiation.cc:407] Connection header received
0806 19:44:23.855395 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:23.855418 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:23.855838 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:23.856267 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:44:23.857690 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:23.858987 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:23.860310 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:23.861076 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:23.861276 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:23.865703 (+ 4427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:44:23.865711 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:44:23.865727 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:44:23.865767 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:44:23.869270 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:23.870350 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:23.870358 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:23.870365 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:23.870549 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:23.871020 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:23.871024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:23.871027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:23.871645 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:44:23.871875 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:23.872496 (+ 621us) server_negotiation.cc:299] Negotiation successful
0806 19:44:23.873026 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":456,"threads_started":1}
I20260806 19:44:23.877173 1431 raft_consensus.cc:348] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e010731dbb5843a3928e5b11e650f9c8" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 33131 } }
I20260806 19:44:23.877590 1431 raft_consensus.cc:374] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:44:23.877700 1431 raft_consensus.cc:729] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e010731dbb5843a3928e5b11e650f9c8, State: Initialized, Role: FOLLOWER
I20260806 19:44:23.878270 1431 consensus_queue.cc:260] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e010731dbb5843a3928e5b11e650f9c8" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 33131 } }
I20260806 19:44:23.878511 1431 raft_consensus.cc:388] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:44:23.878659 1431 raft_consensus.cc:482] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:44:23.878834 1431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:44:23.881435 1431 raft_consensus.cc:504] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e010731dbb5843a3928e5b11e650f9c8" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 33131 } }
I20260806 19:44:23.881938 1431 leader_election.cc:302] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e010731dbb5843a3928e5b11e650f9c8; no voters:
I20260806 19:44:23.882556 1431 leader_election.cc:288] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:44:23.882722 1436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:44:23.883926 1436 raft_consensus.cc:686] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [term 1 LEADER]: Becoming Leader. State: Replica: e010731dbb5843a3928e5b11e650f9c8, State: Running, Role: LEADER
I20260806 19:44:23.884483 1436 consensus_queue.cc:237] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e010731dbb5843a3928e5b11e650f9c8" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 33131 } }
I20260806 19:44:23.884639 1431 sys_catalog.cc:564] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:44:23.887802 1438 sys_catalog.cc:455] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e010731dbb5843a3928e5b11e650f9c8. Latest consensus state: current_term: 1 leader_uuid: "e010731dbb5843a3928e5b11e650f9c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e010731dbb5843a3928e5b11e650f9c8" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 33131 } } }
I20260806 19:44:23.888207 1438 sys_catalog.cc:458] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [sys.catalog]: This master's current role is: LEADER
I20260806 19:44:23.888749 1437 sys_catalog.cc:455] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e010731dbb5843a3928e5b11e650f9c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e010731dbb5843a3928e5b11e650f9c8" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 33131 } } }
I20260806 19:44:23.889077 1437 sys_catalog.cc:458] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8 [sys.catalog]: This master's current role is: LEADER
I20260806 19:44:23.890983 1444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:44:23.896785 1444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:44:23.897836 1431 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 19:44:23.916268 1444 catalog_manager.cc:1347] Generated new cluster ID: 96fd2707587d4542b02582eb3ecf30b2
I20260806 19:44:23.917304 1444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:44:23.956689 1444 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:44:23.958897 1444 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:44:23.977032 1444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8: Generated new TSK 0
I20260806 19:44:23.978315 1444 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:44:24.726199 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.202.84:36943/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:33131
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:44:25 dist-test-slave-77v9 krb5kdc[901](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786045465, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 19:44:25.163101 1476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:44:25.163560 1476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:44:25.163729 1476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:44:25.173131 1476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:44:25.173305 1476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:44:25.173503 1476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:44:25.184314 1476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.202.84:36943/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:33131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:44:25.186066 1476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:44:25.187539 1476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:44:25.205977 1486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:44:25.213011 1490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:44:25.213119 1487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:44:25.214691 1476 server_base.cc:1034] running on GCE node
I20260806 19:44:25.215782 1476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:44:25.217151 1476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:44:25.218622 1476 hybrid_clock.cc:648] HybridClock initialized: now 1786045465218526 us; error 89 us; skew 500 ppm
I20260806 19:44:25.219154 1476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:44:25 dist-test-slave-77v9 krb5kdc[901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045465, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:44:25.223645 1476 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:44:25.230362 1476 webserver.cc:460] Webserver started at http://127.21.202.65:39929/ using document root <none> and password file <none>
I20260806 19:44:25.231395 1476 fs_manager.cc:359] Metadata directory not provided
I20260806 19:44:25.231559 1476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:44:25.231977 1476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:44:25 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045465, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:44:25.255811 1476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "579ff345e8474e708f28bb4ef9584769"
format_stamp: "Formatted at 2026-08-06 19:44:25 on dist-test-slave-77v9"
server_key: "b546691fc5e3edcb05649ae3595f0b41"
server_key_iv: "965d0ab689271778f7932e3c4386402f"
server_key_version: "kuduclusterkey@0"
I20260806 19:44:25.261248 1476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "579ff345e8474e708f28bb4ef9584769"
format_stamp: "Formatted at 2026-08-06 19:44:25 on dist-test-slave-77v9"
server_key: "b546691fc5e3edcb05649ae3595f0b41"
server_key_iv: "965d0ab689271778f7932e3c4386402f"
server_key_version: "kuduclusterkey@0"
I20260806 19:44:25.283587 1476 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.012s sys 0.008s
I20260806 19:44:25.311867 1496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.003s user 0.000s sys 0.000s
I20260806 19:44:25.316298 1476 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.000s sys 0.008s
I20260806 19:44:25.316645 1476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "579ff345e8474e708f28bb4ef9584769"
format_stamp: "Formatted at 2026-08-06 19:44:25 on dist-test-slave-77v9"
server_key: "b546691fc5e3edcb05649ae3595f0b41"
server_key_iv: "965d0ab689271778f7932e3c4386402f"
server_key_version: "kuduclusterkey@0"
I20260806 19:44:25.317093 1476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:44:25.368721 1476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:44:25.374346 1476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:44:25.374738 1476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:44:25.376093 1476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:44:25.378085 1476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:44:25.378180 1476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:44:25.378306 1476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:44:25.378376 1476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:44:25 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045465, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:44:25.917989 1476 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:38337
I20260806 19:44:25.918023 1613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:38337 every 8 connection(s)
I20260806 19:44:25.919559 1476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:44:25.929594 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 1476
I20260806 19:44:25.929844 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:44:25.930857 22313 external_mini_cluster.cc:1428] Setting key 9f6c4335efc9c7e12f4eb0c97375216b
Aug 06 19:44:25 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045465, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:44:25.970896 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:25.923539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:43451 (local address 127.21.202.126:33131)
0806 19:44:25.942870 (+ 19331us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:25.942877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:25.942907 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:44:25.942984 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:25.942990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:25.943096 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:25.943275 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:44:25.946764 (+ 3489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:25.947863 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:25.953635 (+ 5772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:25.954240 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:25.958571 (+ 4331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:25.958811 (+ 240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:44:25.958817 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:44:25.958821 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:44:25.958853 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:44:25.962605 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:25.963778 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:25.963787 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:25.963792 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:25.964037 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:25.964650 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:25.964656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:25.964660 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:25.965103 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:44:25.965255 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:25.970137 (+ 4882us) server_negotiation.cc:299] Negotiation successful
0806 19:44:25.970465 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19168,"thread_start_us":19880,"threads_started":1}
I20260806 19:44:25.974238 1617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:25.924392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:33131 (local address 127.21.202.65:43451)
0806 19:44:25.935831 (+ 11439us) negotiation.cc:107] Waiting for socket to connect
0806 19:44:25.935897 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 19:44:25.939434 (+ 3537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:44:25.943643 (+ 4209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:44:25.943659 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:44:25.944273 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:44:25.946536 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:25.946566 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:25.948051 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:25.948057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:25.953389 (+ 5332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:25.953429 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:25.954509 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:25.954516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:25.954792 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:25.955653 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:44:25.955685 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:44:25.958346 (+ 2661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:44:25.962798 (+ 4452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:44:25.962813 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:44:25.962837 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:44:25.963597 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:44:25.964201 (+ 604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:44:25.964205 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:44:25.964208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:44:25.964498 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:44:25.968991 (+ 4493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:44:25.969000 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:44:25.969570 (+ 570us) client_negotiation.cc:769] Sending connection context
0806 19:44:25.971793 (+ 2223us) client_negotiation.cc:240] Negotiation successful
0806 19:44:25.972448 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11095,"thread_start_us":10836,"threads_started":1}
I20260806 19:44:25.978048 1614 heartbeater.cc:347] Connected to a master server at 127.21.202.126:33131
I20260806 19:44:25.978404 1614 heartbeater.cc:464] Registering TS with master...
I20260806 19:44:25.979403 1614 heartbeater.cc:511] Master 127.21.202.126:33131 requested a full tablet report, sending...
I20260806 19:44:25.982117 1394 ts_manager.cc:194] Registered new tserver with Master: 579ff345e8474e708f28bb4ef9584769 (127.21.202.65:38337)
I20260806 19:44:25.984175 1394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:43451
Aug 06 19:44:25 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045465, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
W20260806 19:44:26.068100 22313 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.202.66: real 0.122s user 0.000s sys 0.024s
I20260806 19:44:26.069286 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.202.84:36943/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:33131
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:44:26.480768 1623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:44:26.481123 1623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:44:26.481315 1623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:44:26.488727 1623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:44:26.488906 1623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:44:26.489063 1623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:44:26.499162 1623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.202.84:36943/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:33131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:44:26.500636 1623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:44:26.502007 1623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:44:26.515758 1637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:44:26.515888 1636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:44:26.515852 1639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:44:26.516737 1623 server_base.cc:1034] running on GCE node
I20260806 19:44:26.517470 1623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:44:26.518499 1623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:44:26.519749 1623 hybrid_clock.cc:648] HybridClock initialized: now 1786045466519679 us; error 47 us; skew 500 ppm
I20260806 19:44:26.520148 1623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:44:26 dist-test-slave-77v9 krb5kdc[901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045466, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:44:26.523818 1623 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:44:26.525511 1623 webserver.cc:460] Webserver started at http://127.21.202.66:39533/ using document root <none> and password file <none>
I20260806 19:44:26.526234 1623 fs_manager.cc:359] Metadata directory not provided
I20260806 19:44:26.526396 1623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:44:26.526737 1623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:44:26 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045466, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:44:26.545425 1623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "62b262a38c324bd0891d13b1b5543b3b"
format_stamp: "Formatted at 2026-08-06 19:44:26 on dist-test-slave-77v9"
server_key: "a46db12712430ed1315877d9bfc387d4"
server_key_iv: "267de153b6c878678fb230761c5b0177"
server_key_version: "kuduclusterkey@0"
I20260806 19:44:26.546670 1623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "62b262a38c324bd0891d13b1b5543b3b"
format_stamp: "Formatted at 2026-08-06 19:44:26 on dist-test-slave-77v9"
server_key: "a46db12712430ed1315877d9bfc387d4"
server_key_iv: "267de153b6c878678fb230761c5b0177"
server_key_version: "kuduclusterkey@0"
I20260806 19:44:26.554075 1623 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 19:44:26.573952 1646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:44:26.575881 1623 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:44:26.576081 1623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "62b262a38c324bd0891d13b1b5543b3b"
format_stamp: "Formatted at 2026-08-06 19:44:26 on dist-test-slave-77v9"
server_key: "a46db12712430ed1315877d9bfc387d4"
server_key_iv: "267de153b6c878678fb230761c5b0177"
server_key_version: "kuduclusterkey@0"
I20260806 19:44:26.576370 1623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:44:26.599047 1623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:44:26.603333 1623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:44:26.603598 1623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:44:26.604549 1623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:44:26.606071 1623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:44:26.606182 1623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:44:26.606344 1623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:44:26.606410 1623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:44:26.643976 1623 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:35471
I20260806 19:44:26.644017 1761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:35471 every 8 connection(s)
I20260806 19:44:26.645336 1623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:44:26.652220 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 1623
I20260806 19:44:26.652413 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:44:26.653064 22313 external_mini_cluster.cc:1428] Setting key 8e479b0d386924fb1b725df395e9adfe
Aug 06 19:44:26 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045466, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:44:26.669210 1764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:26.647946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:48153 (local address 127.21.202.126:33131)
0806 19:44:26.648442 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:26.648451 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:26.649486 (+ 1035us) server_negotiation.cc:407] Connection header received
0806 19:44:26.651198 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:26.651202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:26.651271 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:26.651380 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:44:26.653276 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:26.654249 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:26.656503 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:26.657051 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:26.657167 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:26.661667 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:44:26.661676 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:44:26.661683 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:44:26.661731 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:44:26.665486 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:26.666413 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:26.666427 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:26.666433 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:26.666596 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:26.667111 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:26.667118 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:26.667122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:26.667540 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:44:26.667701 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:26.668410 (+ 709us) server_negotiation.cc:299] Negotiation successful
0806 19:44:26.668669 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":223,"threads_started":1}
I20260806 19:44:26.669865 1765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:26.648506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:33131 (local address 127.21.202.66:48153)
0806 19:44:26.649300 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0806 19:44:26.649351 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 19:44:26.650937 (+ 1586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:44:26.651591 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:44:26.651598 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:44:26.652198 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:44:26.653056 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:26.653084 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:26.654405 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:26.654409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:26.656316 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:26.656344 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:26.657285 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:26.657295 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:26.657646 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:26.658406 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:44:26.658449 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:44:26.661414 (+ 2965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:44:26.665645 (+ 4231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:44:26.665656 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:44:26.665675 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:44:26.666265 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:44:26.666712 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:44:26.666719 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:44:26.666724 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:44:26.666975 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:44:26.667683 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:44:26.667689 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:44:26.668017 (+ 328us) client_negotiation.cc:769] Sending connection context
0806 19:44:26.668243 (+ 226us) client_negotiation.cc:240] Negotiation successful
0806 19:44:26.668655 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":368,"threads_started":1}
I20260806 19:44:26.671514 1762 heartbeater.cc:347] Connected to a master server at 127.21.202.126:33131
I20260806 19:44:26.671845 1762 heartbeater.cc:464] Registering TS with master...
I20260806 19:44:26.672741 1762 heartbeater.cc:511] Master 127.21.202.126:33131 requested a full tablet report, sending...
I20260806 19:44:26.674528 1394 ts_manager.cc:194] Registered new tserver with Master: 62b262a38c324bd0891d13b1b5543b3b (127.21.202.66:35471)
I20260806 19:44:26.675367 1394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:48153
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:44:26.721268 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.202.84:36943/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:33131
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:44:26.987989 1614 heartbeater.cc:503] Master 127.21.202.126:33131 was elected leader, sending a full tablet report...
W20260806 19:44:27.078939 1768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:44:27.079221 1768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:44:27.079383 1768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:44:27.087181 1768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:44:27.087287 1768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:44:27.087471 1768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:44:27.097482 1768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.202.84:36943/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:33131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:44:27.098834 1768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:44:27.100287 1768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:44:27.114168 1774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:44:27.114130 1777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:44:27.114090 1775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:44:27.115198 1768 server_base.cc:1034] running on GCE node
I20260806 19:44:27.115831 1768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:44:27.116781 1768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:44:27.118109 1768 hybrid_clock.cc:648] HybridClock initialized: now 1786045467118060 us; error 52 us; skew 500 ppm
I20260806 19:44:27.118468 1768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:44:27 dist-test-slave-77v9 krb5kdc[901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045467, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:44:27.121661 1768 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:44:27.123222 1768 webserver.cc:460] Webserver started at http://127.21.202.67:41661/ using document root <none> and password file <none>
I20260806 19:44:27.123919 1768 fs_manager.cc:359] Metadata directory not provided
I20260806 19:44:27.124073 1768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:44:27.124323 1768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:44:27 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045467, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:44:27.140028 1768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "8aed22aae84d42a6aa101f82b08388d6"
format_stamp: "Formatted at 2026-08-06 19:44:27 on dist-test-slave-77v9"
server_key: "902822324aa3ca536e5b4b5662047cea"
server_key_iv: "e4d25187c7c6a0116dea191b6a2298dc"
server_key_version: "kuduclusterkey@0"
I20260806 19:44:27.140974 1768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "8aed22aae84d42a6aa101f82b08388d6"
format_stamp: "Formatted at 2026-08-06 19:44:27 on dist-test-slave-77v9"
server_key: "902822324aa3ca536e5b4b5662047cea"
server_key_iv: "e4d25187c7c6a0116dea191b6a2298dc"
server_key_version: "kuduclusterkey@0"
I20260806 19:44:27.146363 1768 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 19:44:27.162784 1784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:44:27.164374 1768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:44:27.164597 1768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "8aed22aae84d42a6aa101f82b08388d6"
format_stamp: "Formatted at 2026-08-06 19:44:27 on dist-test-slave-77v9"
server_key: "902822324aa3ca536e5b4b5662047cea"
server_key_iv: "e4d25187c7c6a0116dea191b6a2298dc"
server_key_version: "kuduclusterkey@0"
I20260806 19:44:27.164892 1768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:44:27.179445 1768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:44:27.183897 1768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:44:27.184252 1768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:44:27.185422 1768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:44:27.187045 1768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:44:27.187129 1768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:44:27.187316 1768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:44:27.187378 1768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:44:27.225373 1768 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:34217
I20260806 19:44:27.225427 1898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:34217 every 8 connection(s)
I20260806 19:44:27.226675 1768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:44:27.233300 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 1768
I20260806 19:44:27.233501 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:44:27.234025 22313 external_mini_cluster.cc:1428] Setting key ba0208186089e0794471617c482e56c0
Aug 06 19:44:27 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045467, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:44:27.248227 1901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.229618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:54745 (local address 127.21.202.126:33131)
0806 19:44:27.230183 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:27.230188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:27.231011 (+ 823us) server_negotiation.cc:407] Connection header received
0806 19:44:27.232948 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:27.232954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:27.233046 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:27.233173 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:44:27.234842 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.235715 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.237491 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.237938 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.238082 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.241622 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:44:27.241628 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:44:27.241632 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:44:27.241669 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:44:27.244595 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:27.245579 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:27.245584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:27.245587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:27.245724 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:27.246242 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:27.246252 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:27.246255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:27.246569 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:44:27.246747 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:27.247499 (+ 752us) server_negotiation.cc:299] Negotiation successful
0806 19:44:27.247777 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":295,"threads_started":1}
I20260806 19:44:27.249334 1902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.230065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:33131 (local address 127.21.202.67:54745)
0806 19:44:27.230819 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0806 19:44:27.230871 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 19:44:27.232636 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:44:27.233448 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:44:27.233461 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:44:27.233959 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:44:27.234654 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:27.234671 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.235871 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.235878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:27.237327 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:27.237346 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.238172 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.238181 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:27.238421 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.239058 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:44:27.239084 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:44:27.241371 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:44:27.244763 (+ 3392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:44:27.244772 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:44:27.244790 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:44:27.245434 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:44:27.245857 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:44:27.245863 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:44:27.245867 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:44:27.246098 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:44:27.246717 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:44:27.246725 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:44:27.247104 (+ 379us) client_negotiation.cc:769] Sending connection context
0806 19:44:27.247397 (+ 293us) client_negotiation.cc:240] Negotiation successful
0806 19:44:27.247956 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":334,"threads_started":1}
I20260806 19:44:27.250945 1899 heartbeater.cc:347] Connected to a master server at 127.21.202.126:33131
I20260806 19:44:27.251291 1899 heartbeater.cc:464] Registering TS with master...
I20260806 19:44:27.252146 1899 heartbeater.cc:511] Master 127.21.202.126:33131 requested a full tablet report, sending...
I20260806 19:44:27.254063 1394 ts_manager.cc:194] Registered new tserver with Master: 8aed22aae84d42a6aa101f82b08388d6 (127.21.202.67:34217)
I20260806 19:44:27.255491 1394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:54745
I20260806 19:44:27.264961 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:44:27.291103 1901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.271744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39700 (local address 127.21.202.126:33131)
0806 19:44:27.271972 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:27.271981 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:27.272652 (+ 671us) server_negotiation.cc:407] Connection header received
0806 19:44:27.272863 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:27.272869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:27.272978 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:27.273107 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:44:27.275257 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.276487 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.278595 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.279287 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.279430 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.283278 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:44:27.283285 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:44:27.283290 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:44:27.283345 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:44:27.287377 (+ 4032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:27.288512 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:27.288521 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:27.288528 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:27.288722 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:27.289212 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:27.289219 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:27.289222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:27.289635 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:44:27.289906 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:27.290506 (+ 600us) server_negotiation.cc:299] Negotiation successful
0806 19:44:27.290735 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 19:44:27.295460 1394 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:44:27.303304 1394 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:44:27.329463 1915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.317170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39346 (local address 127.21.202.66:35471)
0806 19:44:27.319574 (+ 2404us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:27.319582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:27.320643 (+ 1061us) server_negotiation.cc:407] Connection header received
0806 19:44:27.320747 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:27.320757 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:27.321039 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:27.321227 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:44:27.322068 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.323672 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.326964 (+ 3292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.327866 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.328042 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.328227 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:27.328910 (+ 683us) server_negotiation.cc:299] Negotiation successful
0806 19:44:27.329091 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2205,"thread_start_us":2047,"threads_started":1}
I20260806 19:44:27.329552 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.317122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:35471 (local address 127.0.0.1:39346)
0806 19:44:27.320357 (+ 3235us) negotiation.cc:107] Waiting for socket to connect
0806 19:44:27.320381 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:44:27.320508 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:44:27.321476 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:44:27.321481 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:44:27.321505 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:44:27.321874 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:27.321885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.323828 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.323833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:27.326808 (+ 2975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:27.326818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.328372 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.328377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:27.328589 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.328693 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 19:44:27.328883 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 19:44:27.329081 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3077,"mutex_wait_us":31,"thread_start_us":1196,"threads_started":1}
I20260806 19:44:27.330291 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.316349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45186 (local address 127.21.202.67:34217)
0806 19:44:27.317173 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:27.317184 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:27.318170 (+ 986us) server_negotiation.cc:407] Connection header received
0806 19:44:27.318690 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:27.318701 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:27.319126 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:27.319402 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:44:27.320670 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.322307 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.326272 (+ 3965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.327856 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.328000 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.328252 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:27.329831 (+ 1579us) server_negotiation.cc:299] Negotiation successful
0806 19:44:27.330010 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":932,"threads_started":1}
I20260806 19:44:27.331094 1914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.316165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:34217 (local address 127.0.0.1:45186)
0806 19:44:27.317985 (+ 1820us) negotiation.cc:107] Waiting for socket to connect
0806 19:44:27.318048 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 19:44:27.318350 (+ 302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:44:27.319821 (+ 1471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:44:27.319827 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:44:27.319859 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:44:27.320445 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:27.320458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.322494 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.322499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:27.326131 (+ 3632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:27.326146 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.328347 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.328359 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:27.328656 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.328693 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:44:27.330510 (+ 1817us) client_negotiation.cc:240] Negotiation successful
0806 19:44:27.330670 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1529,"mutex_wait_us":44,"thread_start_us":3877,"threads_started":1}
I20260806 19:44:27.333978 1694 tablet_service.cc:1467] Processing CreateTablet for tablet 8e73adf2ae174673a87e58615c5b7f90 (DEFAULT_TABLE table=test-table [id=c593b21ea5094bad9a904c4e30e11449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:44:27.334352 1833 tablet_service.cc:1467] Processing CreateTablet for tablet 8e73adf2ae174673a87e58615c5b7f90 (DEFAULT_TABLE table=test-table [id=c593b21ea5094bad9a904c4e30e11449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:44:27.335510 1694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e73adf2ae174673a87e58615c5b7f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:44:27.335740 1833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e73adf2ae174673a87e58615c5b7f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:44:27.335816 1917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.321640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:38337 (local address 127.0.0.1:40170)
0806 19:44:27.325991 (+ 4351us) negotiation.cc:107] Waiting for socket to connect
0806 19:44:27.326005 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 19:44:27.326164 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:44:27.326940 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:44:27.326945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:44:27.326970 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:44:27.327380 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:27.327391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.329485 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.329492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:27.331132 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:27.331143 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.335027 (+ 3884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.335034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:27.335207 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.335240 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 19:44:27.335338 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 19:44:27.335489 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4231,"thread_start_us":4063,"threads_started":1}
I20260806 19:44:27.336237 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.320800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40170 (local address 127.21.202.65:38337)
0806 19:44:27.325580 (+ 4780us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:27.325586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:27.326117 (+ 531us) server_negotiation.cc:407] Connection header received
0806 19:44:27.326357 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:27.326366 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:27.326719 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:27.327585 (+ 866us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:44:27.327824 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.329308 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.331533 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.334778 (+ 3245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.334918 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.335255 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:27.335381 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 19:44:27.335552 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4499,"thread_start_us":4344,"threads_started":1}
I20260806 19:44:27.340348 1547 tablet_service.cc:1467] Processing CreateTablet for tablet 8e73adf2ae174673a87e58615c5b7f90 (DEFAULT_TABLE table=test-table [id=c593b21ea5094bad9a904c4e30e11449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:44:27.341817 1547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e73adf2ae174673a87e58615c5b7f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:44:27.348326 1918 tablet_bootstrap.cc:492] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b: Bootstrap starting.
I20260806 19:44:27.349927 1919 tablet_bootstrap.cc:492] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6: Bootstrap starting.
I20260806 19:44:27.352056 1918 tablet_bootstrap.cc:654] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b: Neither blocks nor log segments found. Creating new log.
I20260806 19:44:27.353449 1918 log.cc:824] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b: Log is configured to *not* fsync() on all Append() calls
I20260806 19:44:27.353508 1919 tablet_bootstrap.cc:654] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6: Neither blocks nor log segments found. Creating new log.
I20260806 19:44:27.354647 1919 log.cc:824] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6: Log is configured to *not* fsync() on all Append() calls
I20260806 19:44:27.356945 1922 tablet_bootstrap.cc:492] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769: Bootstrap starting.
I20260806 19:44:27.357582 1918 tablet_bootstrap.cc:492] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b: No bootstrap required, opened a new log
I20260806 19:44:27.357810 1919 tablet_bootstrap.cc:492] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6: No bootstrap required, opened a new log
I20260806 19:44:27.357864 1918 ts_tablet_manager.cc:1397] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260806 19:44:27.358081 1919 ts_tablet_manager.cc:1397] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 19:44:27.360617 1922 tablet_bootstrap.cc:654] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769: Neither blocks nor log segments found. Creating new log.
I20260806 19:44:27.361884 1922 log.cc:824] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769: Log is configured to *not* fsync() on all Append() calls
I20260806 19:44:27.365142 1922 tablet_bootstrap.cc:492] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769: No bootstrap required, opened a new log
I20260806 19:44:27.365424 1922 ts_tablet_manager.cc:1397] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 19:44:27.365856 1918 raft_consensus.cc:348] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed22aae84d42a6aa101f82b08388d6" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34217 } } peers { permanent_uuid: "62b262a38c324bd0891d13b1b5543b3b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35471 } } peers { permanent_uuid: "579ff345e8474e708f28bb4ef9584769" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38337 } }
I20260806 19:44:27.366232 1918 raft_consensus.cc:374] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:44:27.366357 1918 raft_consensus.cc:729] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62b262a38c324bd0891d13b1b5543b3b, State: Initialized, Role: FOLLOWER
I20260806 19:44:27.367111 1918 consensus_queue.cc:260] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed22aae84d42a6aa101f82b08388d6" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34217 } } peers { permanent_uuid: "62b262a38c324bd0891d13b1b5543b3b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35471 } } peers { permanent_uuid: "579ff345e8474e708f28bb4ef9584769" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38337 } }
I20260806 19:44:27.368362 1919 raft_consensus.cc:348] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed22aae84d42a6aa101f82b08388d6" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34217 } } peers { permanent_uuid: "62b262a38c324bd0891d13b1b5543b3b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35471 } } peers { permanent_uuid: "579ff345e8474e708f28bb4ef9584769" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38337 } }
I20260806 19:44:27.368876 1919 raft_consensus.cc:374] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:44:27.368983 1919 raft_consensus.cc:729] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aed22aae84d42a6aa101f82b08388d6, State: Initialized, Role: FOLLOWER
I20260806 19:44:27.369714 1762 heartbeater.cc:503] Master 127.21.202.126:33131 was elected leader, sending a full tablet report...
I20260806 19:44:27.369599 1919 consensus_queue.cc:260] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed22aae84d42a6aa101f82b08388d6" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34217 } } peers { permanent_uuid: "62b262a38c324bd0891d13b1b5543b3b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35471 } } peers { permanent_uuid: "579ff345e8474e708f28bb4ef9584769" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38337 } }
I20260806 19:44:27.370412 1918 ts_tablet_manager.cc:1428] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260806 19:44:27.371698 1899 heartbeater.cc:503] Master 127.21.202.126:33131 was elected leader, sending a full tablet report...
I20260806 19:44:27.372336 1919 ts_tablet_manager.cc:1428] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260806 19:44:27.377530 1922 raft_consensus.cc:348] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed22aae84d42a6aa101f82b08388d6" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34217 } } peers { permanent_uuid: "62b262a38c324bd0891d13b1b5543b3b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35471 } } peers { permanent_uuid: "579ff345e8474e708f28bb4ef9584769" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38337 } }
I20260806 19:44:27.377866 1922 raft_consensus.cc:374] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:44:27.378006 1922 raft_consensus.cc:729] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579ff345e8474e708f28bb4ef9584769, State: Initialized, Role: FOLLOWER
I20260806 19:44:27.378592 1922 consensus_queue.cc:260] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed22aae84d42a6aa101f82b08388d6" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34217 } } peers { permanent_uuid: "62b262a38c324bd0891d13b1b5543b3b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35471 } } peers { permanent_uuid: "579ff345e8474e708f28bb4ef9584769" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38337 } }
I20260806 19:44:27.380213 1922 ts_tablet_manager.cc:1428] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769: Time spent starting tablet: real 0.015s user 0.008s sys 0.005s
W20260806 19:44:27.397567 1763 tablet.cc:2369] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:44:27.434002 1615 tablet.cc:2369] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:44:27.478421 1900 tablet.cc:2369] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:44:27.528415 1925 raft_consensus.cc:482] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:44:27.528872 1925 raft_consensus.cc:504] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed22aae84d42a6aa101f82b08388d6" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34217 } } peers { permanent_uuid: "62b262a38c324bd0891d13b1b5543b3b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35471 } } peers { permanent_uuid: "579ff345e8474e708f28bb4ef9584769" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38337 } }
I20260806 19:44:27.530850 1925 leader_election.cc:288] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62b262a38c324bd0891d13b1b5543b3b (127.21.202.66:35471), 579ff345e8474e708f28bb4ef9584769 (127.21.202.65:38337)
I20260806 19:44:27.537185 1915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.531440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:51559 (local address 127.21.202.66:35471)
0806 19:44:27.531710 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:27.531721 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:27.531752 (+ 31us) server_negotiation.cc:407] Connection header received
0806 19:44:27.531867 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:27.531876 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:27.531980 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:27.532104 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:44:27.532626 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.533490 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.535303 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.536172 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.536352 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.536432 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:27.536730 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 19:44:27.536907 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 19:44:27.537190 1902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.531163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:35471 (local address 127.21.202.67:51559)
0806 19:44:27.531497 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 19:44:27.531520 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:44:27.531663 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:44:27.532149 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:44:27.532156 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:44:27.532195 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:44:27.532469 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:27.532481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.533642 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.533650 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:27.535111 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:27.535123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.536338 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.536344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:27.536553 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.536579 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 19:44:27.536697 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 19:44:27.536884 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260806 19:44:27.537508 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.531394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:49621 (local address 127.21.202.65:38337)
0806 19:44:27.531615 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:27.531622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:27.532296 (+ 674us) server_negotiation.cc:407] Connection header received
0806 19:44:27.532562 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:27.532571 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:27.532674 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:27.532802 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:44:27.533375 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.534085 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.535763 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.536556 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.536740 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.536819 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:27.537126 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 19:44:27.537248 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 19:44:27.537925 1928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.531377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:38337 (local address 127.21.202.67:49621)
0806 19:44:27.532181 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0806 19:44:27.532198 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:44:27.532326 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:44:27.532840 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:44:27.532845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:44:27.532870 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:44:27.533203 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:27.533212 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.534245 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.534249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:27.535605 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:44:27.535616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.536770 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.536777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:44:27.536964 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.536987 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 19:44:27.537089 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 19:44:27.537211 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"mutex_wait_us":1,"thread_start_us":465,"threads_started":1}
I20260806 19:44:27.538461 1716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e73adf2ae174673a87e58615c5b7f90" candidate_uuid: "8aed22aae84d42a6aa101f82b08388d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62b262a38c324bd0891d13b1b5543b3b" is_pre_election: true
I20260806 19:44:27.539028 1716 raft_consensus.cc:2454] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aed22aae84d42a6aa101f82b08388d6 in term 0.
I20260806 19:44:27.539032 1568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e73adf2ae174673a87e58615c5b7f90" candidate_uuid: "8aed22aae84d42a6aa101f82b08388d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579ff345e8474e708f28bb4ef9584769" is_pre_election: true
I20260806 19:44:27.539649 1568 raft_consensus.cc:2454] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aed22aae84d42a6aa101f82b08388d6 in term 0.
I20260806 19:44:27.539707 1788 leader_election.cc:302] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62b262a38c324bd0891d13b1b5543b3b, 8aed22aae84d42a6aa101f82b08388d6; no voters:
I20260806 19:44:27.540082 1925 raft_consensus.cc:2781] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:44:27.540241 1925 raft_consensus.cc:482] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:44:27.540359 1925 raft_consensus.cc:3037] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:44:27.542649 1925 raft_consensus.cc:504] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed22aae84d42a6aa101f82b08388d6" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34217 } } peers { permanent_uuid: "62b262a38c324bd0891d13b1b5543b3b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35471 } } peers { permanent_uuid: "579ff345e8474e708f28bb4ef9584769" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38337 } }
I20260806 19:44:27.543321 1925 leader_election.cc:288] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [CANDIDATE]: Term 1 election: Requested vote from peers 62b262a38c324bd0891d13b1b5543b3b (127.21.202.66:35471), 579ff345e8474e708f28bb4ef9584769 (127.21.202.65:38337)
I20260806 19:44:27.543747 1716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e73adf2ae174673a87e58615c5b7f90" candidate_uuid: "8aed22aae84d42a6aa101f82b08388d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62b262a38c324bd0891d13b1b5543b3b"
I20260806 19:44:27.543967 1716 raft_consensus.cc:3037] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:44:27.544023 1568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e73adf2ae174673a87e58615c5b7f90" candidate_uuid: "8aed22aae84d42a6aa101f82b08388d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579ff345e8474e708f28bb4ef9584769"
I20260806 19:44:27.544306 1568 raft_consensus.cc:3037] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:44:27.546376 1716 raft_consensus.cc:2454] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aed22aae84d42a6aa101f82b08388d6 in term 1.
I20260806 19:44:27.546766 1788 leader_election.cc:302] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62b262a38c324bd0891d13b1b5543b3b, 8aed22aae84d42a6aa101f82b08388d6; no voters:
I20260806 19:44:27.547081 1925 raft_consensus.cc:2781] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:44:27.547338 1568 raft_consensus.cc:2454] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aed22aae84d42a6aa101f82b08388d6 in term 1.
I20260806 19:44:27.547756 1925 raft_consensus.cc:686] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [term 1 LEADER]: Becoming Leader. State: Replica: 8aed22aae84d42a6aa101f82b08388d6, State: Running, Role: LEADER
I20260806 19:44:27.548350 1925 consensus_queue.cc:237] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed22aae84d42a6aa101f82b08388d6" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34217 } } peers { permanent_uuid: "62b262a38c324bd0891d13b1b5543b3b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35471 } } peers { permanent_uuid: "579ff345e8474e708f28bb4ef9584769" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38337 } }
I20260806 19:44:27.553560 1394 catalog_manager.cc:5495] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8aed22aae84d42a6aa101f82b08388d6 (127.21.202.67). New cstate: current_term: 1 leader_uuid: "8aed22aae84d42a6aa101f82b08388d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed22aae84d42a6aa101f82b08388d6" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 34217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62b262a38c324bd0891d13b1b5543b3b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579ff345e8474e708f28bb4ef9584769" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38337 } health_report { overall_health: UNKNOWN } } }
I20260806 19:44:27.626734 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.620646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45200 (local address 127.21.202.67:34217)
0806 19:44:27.620926 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:27.620931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:27.620951 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:44:27.621115 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:27.621119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:27.621208 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:27.621345 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:44:27.621726 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.622484 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.624177 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.624662 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.624779 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.625440 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:44:27.625981 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:44:27.626114 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:27.626296 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 19:44:27.626375 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 19:44:27.635516 1568 raft_consensus.cc:1261] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769 [term 1 FOLLOWER]: Refusing update from remote peer 8aed22aae84d42a6aa101f82b08388d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:44:27.635560 1716 raft_consensus.cc:1261] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b [term 1 FOLLOWER]: Refusing update from remote peer 8aed22aae84d42a6aa101f82b08388d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:44:27.636514 1929 consensus_queue.cc:1035] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62b262a38c324bd0891d13b1b5543b3b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 35471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:44:27.636909 1925 consensus_queue.cc:1035] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "579ff345e8474e708f28bb4ef9584769" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:44:27.648514 1932 mvcc.cc:205] Tried to move back new op lower bound from 7315642235424743424 to 7315642235082928128. Current Snapshot: MvccSnapshot[applied={T|T < 7315642235424743424}]
I20260806 19:44:27.669819 1393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:44:27.670179 1393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:44:27.674657 1393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e010731dbb5843a3928e5b11e650f9c8: Sending DeleteTablet for 3 replicas of tablet 8e73adf2ae174673a87e58615c5b7f90
I20260806 19:44:27.676041 1694 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e73adf2ae174673a87e58615c5b7f90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:44:27 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:39346
I20260806 19:44:27.676697 1833 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e73adf2ae174673a87e58615c5b7f90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:44:27 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:45186
I20260806 19:44:27.676705 1693 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e73adf2ae174673a87e58615c5b7f90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:44:27 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:39346
I20260806 19:44:27.677019 1547 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e73adf2ae174673a87e58615c5b7f90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:44:27 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:40170
I20260806 19:44:27.678589 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 1476
I20260806 19:44:27.705806 1944 tablet_replica.cc:330] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769: stopping tablet replica
I20260806 19:44:27.709769 1944 raft_consensus.cc:2229] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:44:27.710731 1944 raft_consensus.cc:2258] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:44:27.717243 1942 tablet_replica.cc:330] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b: stopping tablet replica
I20260806 19:44:27.717717 1942 raft_consensus.cc:2229] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:44:27.718254 1942 raft_consensus.cc:2258] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:44:27.719209 1943 tablet_replica.cc:330] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6: stopping tablet replica
I20260806 19:44:27.719695 1943 raft_consensus.cc:2229] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:44:27.720496 1943 raft_consensus.cc:2258] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:44:27.721105 1942 ts_tablet_manager.cc:1905] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:44:27.722761 1944 ts_tablet_manager.cc:1905] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:44:27.726794 1943 ts_tablet_manager.cc:1905] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:44:27.729008 1942 ts_tablet_manager.cc:1918] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:44:27.729182 1942 log.cc:1196] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/8e73adf2ae174673a87e58615c5b7f90
I20260806 19:44:27.729761 1942 ts_tablet_manager.cc:1939] T 8e73adf2ae174673a87e58615c5b7f90 P 62b262a38c324bd0891d13b1b5543b3b: Deleting consensus metadata
I20260806 19:44:27.731194 1944 ts_tablet_manager.cc:1918] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:44:27.731395 1944 log.cc:1196] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/8e73adf2ae174673a87e58615c5b7f90
I20260806 19:44:27.731866 1381 catalog_manager.cc:4841] TS 62b262a38c324bd0891d13b1b5543b3b (127.21.202.66:35471): tablet 8e73adf2ae174673a87e58615c5b7f90 (table test-table [id=c593b21ea5094bad9a904c4e30e11449]) successfully deleted
I20260806 19:44:27.731973 1944 ts_tablet_manager.cc:1939] T 8e73adf2ae174673a87e58615c5b7f90 P 579ff345e8474e708f28bb4ef9584769: Deleting consensus metadata
W20260806 19:44:27.733152 1381 catalog_manager.cc:4808] TS 62b262a38c324bd0891d13b1b5543b3b (127.21.202.66:35471): delete failed for tablet 8e73adf2ae174673a87e58615c5b7f90 because the tablet was not found. No further retry: Not found: Tablet not found: 8e73adf2ae174673a87e58615c5b7f90
I20260806 19:44:27.734064 1380 catalog_manager.cc:4841] TS 579ff345e8474e708f28bb4ef9584769 (127.21.202.65:38337): tablet 8e73adf2ae174673a87e58615c5b7f90 (table test-table [id=c593b21ea5094bad9a904c4e30e11449]) successfully deleted
I20260806 19:44:27.734841 1943 ts_tablet_manager.cc:1918] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:44:27.735016 1943 log.cc:1196] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/8e73adf2ae174673a87e58615c5b7f90
I20260806 19:44:27.735584 1943 ts_tablet_manager.cc:1939] T 8e73adf2ae174673a87e58615c5b7f90 P 8aed22aae84d42a6aa101f82b08388d6: Deleting consensus metadata
I20260806 19:44:27.737231 1380 catalog_manager.cc:4841] TS 8aed22aae84d42a6aa101f82b08388d6 (127.21.202.67:34217): tablet 8e73adf2ae174673a87e58615c5b7f90 (table test-table [id=c593b21ea5094bad9a904c4e30e11449]) successfully deleted
Aug 06 19:44:27 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:44:27.747169 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.681028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40178 (local address 127.21.202.65:38337)
0806 19:44:27.681238 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:27.681244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:27.703916 (+ 22672us) server_negotiation.cc:407] Connection header received
0806 19:44:27.706184 (+ 2268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:27.706194 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:27.706329 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:27.706469 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:44:27.711652 (+ 5183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.713316 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.724461 (+ 11145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.725248 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.725370 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.740289 (+ 14919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:44:27.740308 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:44:27.740327 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:44:27.740382 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:44:27.744079 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:27.744851 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:27.744863 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:27.744866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:27.745028 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:27.745414 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:27.745419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:27.745422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:27.745896 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:44:27.746187 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:27.746488 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 19:44:27.746750 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 19:44:27.747865 1608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:44:27.841346 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 1476
I20260806 19:44:27.862340 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 1623
Aug 06 19:44:27 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:44:27.882659 1915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:27.863235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39348 (local address 127.21.202.66:35471)
0806 19:44:27.863438 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:27.863445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:27.863543 (+ 98us) server_negotiation.cc:407] Connection header received
0806 19:44:27.863747 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:27.863753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:27.863844 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:27.863951 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:44:27.865640 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.866777 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.868774 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:27.869506 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:27.869614 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:27.875146 (+ 5532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:44:27.875166 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:44:27.875189 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:44:27.875244 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:44:27.879264 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:27.880174 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:27.880186 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:27.880189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:27.880378 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:27.880736 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:27.880741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:27.880744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:27.881315 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:44:27.881622 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:27.881998 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 19:44:27.882258 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 19:44:27.883486 1757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:44:27.976686 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 1623
I20260806 19:44:28.002399 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 1768
Aug 06 19:44:28 dist-test-slave-77v9 krb5kdc[901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:44:28.018654 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:44:28.003251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45214 (local address 127.21.202.67:34217)
0806 19:44:28.003468 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 19:44:28.003474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:44:28.003498 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:44:28.003670 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:44:28.003675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:44:28.003750 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:44:28.003868 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:44:28.005183 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:28.006006 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:28.007300 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:44:28.007769 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:44:28.007929 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:44:28.012441 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:44:28.012455 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:44:28.012471 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:44:28.012509 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:44:28.015696 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:28.016401 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:28.016414 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:28.016417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:28.016549 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:44:28.017013 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:44:28.017018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:44:28.017023 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:44:28.017377 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:44:28.017722 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0806 19:44:28.018052 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 19:44:28.018279 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 19:44:28.019333 1894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:44:28.143857 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 1768
I20260806 19:44:28.165992 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 1361
I20260806 19:44:28.167284 1426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:44:28.281706 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 1361
I20260806 19:44:28.306694 22313 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 19:44:28.653016 22313 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 19:44:28.653331 22313 mini_ranger.cc:66] Stopping Ranger...
I20260806 19:44:28.999464 22313 mini_ranger.cc:68] Stopped Ranger
2026-08-06 19:44:28.999 UTC [932] LOG: received smart shutdown request
2026-08-06 19:44:29.004 UTC [932] LOG: background worker "logical replication launcher" (PID 941) exited with exit code 1
2026-08-06 19:44:29.004 UTC [936] LOG: shutting down
2026-08-06 19:44:29.187 UTC [932] LOG: database system is shut down
2026-08-06T19:44:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53907 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:44:29 dist-test-slave-77v9 krb5kdc[1958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:44:29 dist-test-slave-77v9 krb5kdc[1958](info): set up 2 sockets
Aug 06 19:44:29 dist-test-slave-77v9 krb5kdc[1958](info): commencing operation
krb5kdc: starting...
W20260806 19:44:30.481432 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.006s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:44:30 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:44:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:44:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/rangeradmin_127.21.202.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/rangeradmin_127.21.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/rangerlookup_127.21.202.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/rangerlookup_127.21.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/HTTP_127.21.202.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/HTTP_127.21.202.84@KRBTEST.COM.keytab.
I20260806 19:44:30.900251 22313 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskha9koQ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/postgres -l logfile start
2026-08-06 19:44:33.884 UTC [1989] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 19:44:33.884 UTC [1989] LOG: listening on IPv4 address "127.21.202.84", port 52735
2026-08-06 19:44:33.889 UTC [1989] LOG: listening on Unix socket "/tmp/.s.PGSQL.52735"
2026-08-06 19:44:33.915 UTC [1992] LOG: database system was shut down at 2026-08-06 19:44:32 UTC
2026-08-06 19:44:33.919 UTC [1989] LOG: database system is ready to accept connections
I20260806 19:44:34.961277 22313 mini_postgres.cc:96] Postgres bound to 52735
2026-08-06 19:44:34.985 UTC [2000] FATAL: database "slave" does not exist
127.21.202.84:52735 - accepting connections
I20260806 19:44:34.986335 22313 mini_ranger.cc:161] Starting Ranger...
I20260806 19:44:35.020418 22313 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 19:44:35.786180 22313 mini_ranger.cc:87] Created ranger Postgres database
I20260806 19:44:35.786341 22313 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-admin
2026-08-06 19:44:36,115 [I] DB FLAVOR :POSTGRES
2026-08-06 19:44:36,116 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 19:44:36,116 [I] Checking connection..
2026-08-06 19:44:36,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 19:44:36,379 [I] Checking connection passed.
2026-08-06 19:44:36,380 [I] --------- Verifying version history table ---------
2026-08-06 19:44:36,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 19:44:36,653 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 19:44:36,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 19:44:36,908 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 19:44:36,908 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 19:44:36,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 19:44:37.158 UTC [2066] WARNING: there is no transaction in progress
2026-08-06 19:44:37,166 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 19:44:37,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 19:44:37,480 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 19:44:37,481 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 19:44:37,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 19:44:37,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 19:44:38,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-77v9.c.gcp-upstream.internal','N') ;"
2026-08-06 19:44:38,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 19:44:38,513 [I] Table x_portal_user does not exist in database ranger
2026-08-06 19:44:38,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 19:44:38,790 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 19:44:38,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 19:44:39,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 19:44:39,301 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 19:44:39,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 19:44:40.385 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.404 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.423 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.440 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.460 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.479 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.541 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.549 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.566 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.578 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.589 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.604 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.614 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.621 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.629 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:40.636 UTC [2179] WARNING: there is no transaction in progress
2026-08-06 19:44:41,385 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 19:44:41,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 19:44:41,632 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 19:44:41,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 19:44:41,905 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 19:44:41,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 19:44:42,181 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 19:44:42,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 19:44:42,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 19:44:42,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-77v9.c.gcp-upstream.internal';"
2026-08-06 19:44:42,948 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 19:44:42,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 19:44:43,216 [I] DB_PATCHES have already been applied
I20260806 19:44:43.222947 22313 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-admin:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskha9koQ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 19:44:43.223070 22313 mini_ranger.cc:193] Using host: 127.21.202.84
I20260806 19:44:43.245038 22313 mini_ranger.cc:227] Ranger admin URL: http://127.21.202.84:35191
Aug 06, 2026 7:44:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 7:44:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 7:44:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 7:44:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 7:44:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 7:44:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 7:44:44 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 19:44:44.446494 22313 mini_ranger.cc:160] Time spent starting Ranger: real 9.460s user 0.000s sys 0.064s
Aug 06, 2026 7:45:06 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 7:45:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 7:45:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 7:45:07 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 7:45:07 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 19:45:08.568002 22313 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/rangerkms_127.21.202.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/rangerkms_127.21.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.202.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.202.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.202.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/HTTP_127.21.202.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.202.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/HTTP_127.21.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 19:45:08 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045508, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 19:45:08.782469 22313 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 19:45:08.810503 22313 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 19:45:09.987330 22313 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 19:45:09.987463 22313 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-kms
I20260806 19:45:09.987521 22313 mini_ranger_kms.cc:227] Using postgres at 127.21.202.84:52735
2026-08-06 19:45:10,431 [I] DB FLAVOR :POSTGRES
2026-08-06 19:45:10,431 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 19:45:10,431 [I] Checking connection
2026-08-06 19:45:10,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 19:45:10,726 [I] connection success
2026-08-06 19:45:10,727 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 19:45:10,727 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 19:45:10,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 19:45:11,026 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 19:45:11,026 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 19:45:11,027 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 19:45:11,027 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.202.84:52735/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 19:45:11,343 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 19:45:11.863663 22313 mini_ranger_kms.cc:310] Created kms service
I20260806 19:45:12.076076 22313 mini_ranger_kms.cc:326] Created kudu user
I20260806 19:45:12.128567 22313 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 19:45:12 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045508, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:45:12.569173 22313 mini_ranger_kms.cc:384] Added ranger policy
I20260806 19:45:12.569535 22313 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-kms:/tmp/dist-test-taskha9koQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskha9koQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 19:45:12.569608 22313 mini_ranger_kms.cc:240] Using host: 127.21.202.84
I20260806 19:45:12.592145 22313 mini_ranger_kms.cc:276] Ranger KMS PID: 2367
I20260806 19:45:12.592314 22313 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.202.84:41071
Aug 06, 2026 7:45:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 7:45:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 7:45:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 7:45:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 7:45:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 7:45:14 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 19:45:14.972730 22313 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.190s user 0.072s sys 0.405s
I20260806 19:45:14.973047 22313 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 19:45:14.973254 22313 mini_ranger_kms.cc:401] 127.21.202.84:41071/kms/v1/keys
Aug 06 19:45:15 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786045515, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.202.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 19:45:15 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045515, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.202.84@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
Aug 06 19:45:15 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045515, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.202.84@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
Aug 06 19:45:18 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:45:18.659524 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42035
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:42035
--ranger_config_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.202.84:41071/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:45:19.061489 2418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:19.061825 2418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:19.061990 2418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:19.070009 2418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:45:19.070147 2418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:19.070252 2418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:19.070358 2418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:45:19.070442 2418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:45:19.083916 2418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.202.84:41071/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:42035
--ranger_config_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:19.085580 2418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:19.087091 2418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:19.102136 2427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:19.102458 2424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:19.102110 2425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:19.103617 2418 server_base.cc:1034] running on GCE node
I20260806 19:45:19.104522 2418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:19.107151 2418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:19.108525 2418 hybrid_clock.cc:648] HybridClock initialized: now 1786045519108436 us; error 80 us; skew 500 ppm
I20260806 19:45:19.109086 2418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:19 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045519, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:19.113113 2418 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:45:19.114969 2418 webserver.cc:460] Webserver started at http://127.21.202.126:45549/ using document root <none> and password file <none>
I20260806 19:45:19.115692 2418 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:19.115846 2418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:19.116197 2418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:45:19 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045519, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:45:19.187840 2418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "4612ddd609204b53ac0f93ff95a8bc5d"
format_stamp: "Formatted at 2026-08-06 19:45:19 on dist-test-slave-77v9"
server_key: "4357937988bcaaea731c7828bbea8363"
server_key_iv: "78ac967efc781f15756548251faf2edc"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:19.189203 2418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "4612ddd609204b53ac0f93ff95a8bc5d"
format_stamp: "Formatted at 2026-08-06 19:45:19 on dist-test-slave-77v9"
server_key: "4357937988bcaaea731c7828bbea8363"
server_key_iv: "78ac967efc781f15756548251faf2edc"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:19.197012 2418 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 19:45:19.219410 2435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:19.221479 2418 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 19:45:19.221735 2418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "4612ddd609204b53ac0f93ff95a8bc5d"
format_stamp: "Formatted at 2026-08-06 19:45:19 on dist-test-slave-77v9"
server_key: "4357937988bcaaea731c7828bbea8363"
server_key_iv: "78ac967efc781f15756548251faf2edc"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:19.222059 2418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:19.251493 2418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:19.257042 2418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:19.257452 2418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:19.284621 2418 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:42035
I20260806 19:45:19.284633 2487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:42035 every 8 connection(s)
I20260806 19:45:19.286442 2418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:45:19.289986 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 2418
I20260806 19:45:19.290210 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:45:19.290975 2488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:19.291127 22313 external_mini_cluster.cc:1428] Setting key 697db953a29680c05936520291c0a949
I20260806 19:45:19.311236 2488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d: Bootstrap starting.
I20260806 19:45:19.316004 2488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:19.317570 2488 log.cc:824] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:19.321305 2488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d: No bootstrap required, opened a new log
Aug 06 19:45:19 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:19.331168 2488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42035 } }
I20260806 19:45:19.331704 2488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:19.331827 2488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4612ddd609204b53ac0f93ff95a8bc5d, State: Initialized, Role: FOLLOWER
I20260806 19:45:19.332602 2488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42035 } }
I20260806 19:45:19.332917 2488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:45:19.333076 2488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:45:19.333287 2488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:19.336594 2491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:19.293557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60624 (local address 127.21.202.126:42035)
0806 19:45:19.309609 (+ 16052us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:19.309629 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:19.309689 (+ 60us) server_negotiation.cc:407] Connection header received
0806 19:45:19.312315 (+ 2626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:19.312348 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:19.312972 (+ 624us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:19.313554 (+ 582us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:19.315229 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:19.317568 (+ 2339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:19.319648 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:19.320747 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:19.321006 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:19.326337 (+ 5331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:19.326351 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:19.326374 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:19.326440 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:19.330939 (+ 4499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:19.332103 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:19.332113 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:19.332122 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:19.332332 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:19.332798 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:19.332834 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:19.332838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:19.333752 (+ 914us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:19.334089 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:19.334608 (+ 519us) server_negotiation.cc:299] Negotiation successful
0806 19:45:19.335090 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15740,"thread_start_us":15499,"threads_started":1}
I20260806 19:45:19.336876 2488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42035 } }
I20260806 19:45:19.337538 2488 leader_election.cc:302] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4612ddd609204b53ac0f93ff95a8bc5d; no voters:
I20260806 19:45:19.338279 2488 leader_election.cc:288] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:45:19.338443 2493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:45:19.342213 2493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 1 LEADER]: Becoming Leader. State: Replica: 4612ddd609204b53ac0f93ff95a8bc5d, State: Running, Role: LEADER
I20260806 19:45:19.342934 2488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:45:19.343209 2493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42035 } }
I20260806 19:45:19.349682 2495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4612ddd609204b53ac0f93ff95a8bc5d. Latest consensus state: current_term: 1 leader_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42035 } } }
I20260806 19:45:19.350126 2495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:19.350718 2494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42035 } } }
I20260806 19:45:19.351006 2494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:19.358433 2500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:45:19.365630 2500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:45:19.366531 2488 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 19:45:19.412395 2500 catalog_manager.cc:1347] Generated new cluster ID: 89326ad3e2774d02abeacd1a37200bff
I20260806 19:45:19.412599 2500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:45:19.440444 2500 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:45:19.442798 2500 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:45:19.454948 2500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d: Generated new TSK 0
I20260806 19:45:19.456305 2500 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:45:20.455186 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.202.84:41071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42035
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:45:20 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786045520, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 19:45:21.019603 2533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:21.020149 2533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:21.020350 2533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:21.033217 2533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:21.033396 2533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:21.033658 2533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:45:21.050496 2533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.202.84:41071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:21.052495 2533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:21.054425 2533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:21.076277 2543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:21.076792 2544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:21.076398 2546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:21.077471 2533 server_base.cc:1034] running on GCE node
I20260806 19:45:21.078598 2533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:21.080793 2533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:21.082203 2533 hybrid_clock.cc:648] HybridClock initialized: now 1786045521082131 us; error 66 us; skew 500 ppm
I20260806 19:45:21.082636 2533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:21 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045521, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:21.087689 2533 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:45:21.090008 2533 webserver.cc:460] Webserver started at http://127.21.202.65:33551/ using document root <none> and password file <none>
I20260806 19:45:21.090840 2533 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:21.091022 2533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:21.091377 2533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:45:21 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045521, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:45:21.121548 2533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "7d39b9875aa547e5b9d82f854e08c0f0"
format_stamp: "Formatted at 2026-08-06 19:45:21 on dist-test-slave-77v9"
server_key: "a9b2ce2c141c252941e7800c82e62002"
server_key_iv: "a490bf8c1602271f5f70ab392f663567"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:21.122920 2533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "7d39b9875aa547e5b9d82f854e08c0f0"
format_stamp: "Formatted at 2026-08-06 19:45:21 on dist-test-slave-77v9"
server_key: "a9b2ce2c141c252941e7800c82e62002"
server_key_iv: "a490bf8c1602271f5f70ab392f663567"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:21.154400 2533 fs_manager.cc:691] Time spent creating directory manager: real 0.031s user 0.016s sys 0.005s
I20260806 19:45:21.188992 2554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:21.193775 2533 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.006s sys 0.001s
I20260806 19:45:21.194159 2533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "7d39b9875aa547e5b9d82f854e08c0f0"
format_stamp: "Formatted at 2026-08-06 19:45:21 on dist-test-slave-77v9"
server_key: "a9b2ce2c141c252941e7800c82e62002"
server_key_iv: "a490bf8c1602271f5f70ab392f663567"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:21.194571 2533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:21.244583 2533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:21.252861 2533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:21.253373 2533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:21.254928 2533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:21.257246 2533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:21.257427 2533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:21.257973 2533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:21.258267 2533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:45:21 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045520, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:45:21.454356 2533 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:37031
I20260806 19:45:21.454407 2672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:37031 every 8 connection(s)
I20260806 19:45:21.456202 2533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:45:21.463686 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 2533
I20260806 19:45:21.463961 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:45:21.465116 22313 external_mini_cluster.cc:1428] Setting key 8398e4063e360f036bcdaa26a8cc0a28
Aug 06 19:45:21 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045521, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:21.496961 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:21.461922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:37901 (local address 127.21.202.126:42035)
0806 19:45:21.475324 (+ 13402us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:21.475331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:21.475356 (+ 25us) server_negotiation.cc:407] Connection header received
0806 19:45:21.475442 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:21.475447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:21.475543 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:21.475668 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:21.478134 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:21.479396 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:21.482200 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:21.482911 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:21.483039 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:21.488341 (+ 5302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:21.488349 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:21.488354 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:21.488421 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:21.492500 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:21.493761 (+ 1261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:21.493768 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:21.493772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:21.493946 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:21.494603 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:21.494609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:21.494615 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:21.495025 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:21.495171 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:21.496266 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0806 19:45:21.496517 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13245,"thread_start_us":14899,"threads_started":1}
I20260806 19:45:21.499676 2675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:21.460897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42035 (local address 127.21.202.65:37901)
0806 19:45:21.469751 (+ 8854us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:21.469848 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:21.473620 (+ 3772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:21.476166 (+ 2546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:21.476182 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:21.476905 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:21.477856 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:21.477888 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:21.479666 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:21.479671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:21.481879 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:21.481898 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:21.483480 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:21.483488 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:21.483813 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:21.484734 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:21.484767 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:21.487945 (+ 3178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:21.492792 (+ 4847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:21.492829 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:21.492852 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:21.493582 (+ 730us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:21.494104 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:21.494109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:21.494113 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:21.494440 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:21.495295 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:21.495303 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:21.495771 (+ 468us) client_negotiation.cc:769] Sending connection context
0806 19:45:21.497580 (+ 1809us) client_negotiation.cc:240] Negotiation successful
0806 19:45:21.498189 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8313,"thread_start_us":8407,"threads_started":1}
I20260806 19:45:21.502228 2673 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42035
I20260806 19:45:21.502604 2673 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:21.503917 2673 heartbeater.cc:511] Master 127.21.202.126:42035 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
I20260806 19:45:21.507139 2452 ts_manager.cc:194] Registered new tserver with Master: 7d39b9875aa547e5b9d82f854e08c0f0 (127.21.202.65:37031)
I20260806 19:45:21.509418 2452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:37901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:45:21.558817 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.202.84:41071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42035
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:45:21 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045520, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
W20260806 19:45:22.137229 2679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:22.137729 2679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:22.137913 2679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:22.150987 2679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:22.151178 2679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:22.151360 2679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:45:22.168684 2679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.202.84:41071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:22.170773 2679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:22.172598 2679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:22.189564 2694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:22.189522 2695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:22.189493 2697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:22.191038 2679 server_base.cc:1034] running on GCE node
I20260806 19:45:22.191833 2679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:22.193153 2679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:22.194430 2679 hybrid_clock.cc:648] HybridClock initialized: now 1786045522194377 us; error 49 us; skew 500 ppm
I20260806 19:45:22.194859 2679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:22 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045522, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:22.199085 2679 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:45:22.203060 2679 webserver.cc:460] Webserver started at http://127.21.202.66:40569/ using document root <none> and password file <none>
I20260806 19:45:22.204156 2679 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:22.204399 2679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:22.205132 2679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:45:22 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045522, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:45:22.223845 2679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "5bc4e48af34842328e525c271e7a575e"
format_stamp: "Formatted at 2026-08-06 19:45:22 on dist-test-slave-77v9"
server_key: "1ded3ef5301a09406554b9c8142246df"
server_key_iv: "912d27882437c455fcd3988123782994"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:22.225067 2679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "5bc4e48af34842328e525c271e7a575e"
format_stamp: "Formatted at 2026-08-06 19:45:22 on dist-test-slave-77v9"
server_key: "1ded3ef5301a09406554b9c8142246df"
server_key_iv: "912d27882437c455fcd3988123782994"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:22.233309 2679 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260806 19:45:22.249722 2704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:22.251966 2679 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:45:22.252254 2679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "5bc4e48af34842328e525c271e7a575e"
format_stamp: "Formatted at 2026-08-06 19:45:22 on dist-test-slave-77v9"
server_key: "1ded3ef5301a09406554b9c8142246df"
server_key_iv: "912d27882437c455fcd3988123782994"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:22.252568 2679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:22.285770 2679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:22.291627 2679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:22.292001 2679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:22.293327 2679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:22.295373 2679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:22.295475 2679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:22.295595 2679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:22.295663 2679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:22.342324 2679 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:45627
I20260806 19:45:22.342407 2824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:45627 every 8 connection(s)
I20260806 19:45:22.344245 2679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:45:22.354243 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 2679
I20260806 19:45:22.354454 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:45:22.354995 22313 external_mini_cluster.cc:1428] Setting key 37c714df1a30236a4f7e93e23e086cf5
Aug 06 19:45:22 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045522, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:22.372272 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:22.348323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42035 (local address 127.21.202.66:52603)
0806 19:45:22.349251 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:22.349303 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:22.351066 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:22.351900 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:22.351909 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:22.352502 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:22.353237 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:22.353259 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:22.355082 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:22.355088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:22.357234 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:22.357265 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:22.358532 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:22.358545 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:22.358977 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:22.359973 (+ 996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:22.360010 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:22.363091 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:22.367458 (+ 4367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:22.367470 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:22.367500 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:22.368121 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:22.368655 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:22.368662 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:22.368668 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:22.369028 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:22.369757 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:22.369767 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:22.370247 (+ 480us) client_negotiation.cc:769] Sending connection context
0806 19:45:22.370539 (+ 292us) client_negotiation.cc:240] Negotiation successful
0806 19:45:22.371134 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":416,"threads_started":1}
I20260806 19:45:22.373508 2827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:22.347684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:52603 (local address 127.21.202.126:42035)
0806 19:45:22.348219 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:22.348226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:22.349464 (+ 1238us) server_negotiation.cc:407] Connection header received
0806 19:45:22.351409 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:22.351416 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:22.351527 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:22.351671 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:22.353496 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:22.354847 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:22.357500 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:22.358078 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:22.358267 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:22.363459 (+ 5192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:22.363465 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:22.363469 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:22.363510 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:22.367184 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:22.368290 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:22.368297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:22.368301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:22.368510 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:22.369166 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:22.369174 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:22.369179 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:22.369555 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:22.369688 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:22.372956 (+ 3268us) server_negotiation.cc:299] Negotiation successful
0806 19:45:22.373223 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":258,"threads_started":1}
I20260806 19:45:22.374918 2825 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42035
I20260806 19:45:22.375258 2825 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:22.376331 2825 heartbeater.cc:511] Master 127.21.202.126:42035 requested a full tablet report, sending...
I20260806 19:45:22.378916 2452 ts_manager.cc:194] Registered new tserver with Master: 5bc4e48af34842328e525c271e7a575e (127.21.202.66:45627)
I20260806 19:45:22.380156 2452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:52603
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:45:22.440528 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.202.84:41071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42035
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:45:22.515311 2673 heartbeater.cc:503] Master 127.21.202.126:42035 was elected leader, sending a full tablet report...
W20260806 19:45:22.863441 2831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:22.863847 2831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:22.864032 2831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:22.872411 2831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:22.872602 2831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:22.872833 2831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:45:22.884472 2831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.202.84:41071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:22.886621 2831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:22.888434 2831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:22.906034 2836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:22.906060 2838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:22.906649 2840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:22.907127 2831 server_base.cc:1034] running on GCE node
I20260806 19:45:22.907876 2831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:22.909150 2831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:22.910454 2831 hybrid_clock.cc:648] HybridClock initialized: now 1786045522910410 us; error 39 us; skew 500 ppm
I20260806 19:45:22.910843 2831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:22 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045522, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:22.922708 2831 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:45:22.925267 2831 webserver.cc:460] Webserver started at http://127.21.202.67:40843/ using document root <none> and password file <none>
I20260806 19:45:22.926374 2831 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:22.926592 2831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:22.927173 2831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:45:22 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045522, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:45:22.947140 2831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "b344c165f1164c868f060c933f4e151b"
format_stamp: "Formatted at 2026-08-06 19:45:22 on dist-test-slave-77v9"
server_key: "e9f076dc4395ae206f490a3601b4f8b8"
server_key_iv: "10c023be41d609fe1d5bdbeb8b6e8834"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:22.948267 2831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "b344c165f1164c868f060c933f4e151b"
format_stamp: "Formatted at 2026-08-06 19:45:22 on dist-test-slave-77v9"
server_key: "e9f076dc4395ae206f490a3601b4f8b8"
server_key_iv: "10c023be41d609fe1d5bdbeb8b6e8834"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:22.954900 2831 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 19:45:22.982730 2847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:22.985184 2831 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:45:22.985435 2831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "b344c165f1164c868f060c933f4e151b"
format_stamp: "Formatted at 2026-08-06 19:45:22 on dist-test-slave-77v9"
server_key: "e9f076dc4395ae206f490a3601b4f8b8"
server_key_iv: "10c023be41d609fe1d5bdbeb8b6e8834"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:22.985889 2831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:23.015712 2831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:23.024052 2831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:23.024474 2831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:23.026064 2831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:23.027851 2831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:23.027944 2831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:23.028091 2831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:23.028164 2831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:23.078859 2831 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:40793
I20260806 19:45:23.078902 2962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:40793 every 8 connection(s)
I20260806 19:45:23.080427 2831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:45:23.084525 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 2831
I20260806 19:45:23.084746 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:45:23.085384 22313 external_mini_cluster.cc:1428] Setting key c3da5cf669bf840a4563201c2b9ed292
Aug 06 19:45:23 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045522, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:23.117229 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:23.085275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:42861 (local address 127.21.202.126:42035)
0806 19:45:23.085962 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:23.085969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:23.087878 (+ 1909us) server_negotiation.cc:407] Connection header received
0806 19:45:23.090452 (+ 2574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:23.090456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:23.090552 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:23.090654 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:23.093625 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:23.094477 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:23.097837 (+ 3360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:23.098872 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:23.099418 (+ 546us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:23.107337 (+ 7919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:23.107346 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:23.107350 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:23.107419 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:23.111455 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:23.112974 (+ 1519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:23.112982 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:23.112985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:23.113263 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:23.114142 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:23.114153 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:23.114157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:23.114584 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:23.114741 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:23.116161 (+ 1420us) server_negotiation.cc:299] Negotiation successful
0806 19:45:23.116770 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":336,"threads_started":1}
I20260806 19:45:23.117843 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:23.086162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42035 (local address 127.21.202.67:42861)
0806 19:45:23.087606 (+ 1444us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:23.087686 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:23.090094 (+ 2408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:23.091206 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:23.091217 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:23.092223 (+ 1006us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:23.093315 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:23.093350 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:23.094749 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:23.094755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:23.097262 (+ 2507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:23.097304 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:23.099704 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:23.099716 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:23.100642 (+ 926us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:23.102184 (+ 1542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:23.102214 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:23.105793 (+ 3579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:23.111702 (+ 5909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:23.111713 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:23.111732 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:23.112494 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:23.113452 (+ 958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:23.113455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:23.113458 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:23.113723 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:23.114762 (+ 1039us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:23.114770 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:23.115316 (+ 546us) client_negotiation.cc:769] Sending connection context
0806 19:45:23.115707 (+ 391us) client_negotiation.cc:240] Negotiation successful
0806 19:45:23.116345 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1048,"thread_start_us":677,"threads_started":1}
I20260806 19:45:23.119908 2963 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42035
I20260806 19:45:23.120232 2963 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:23.121119 2963 heartbeater.cc:511] Master 127.21.202.126:42035 requested a full tablet report, sending...
I20260806 19:45:23.123298 2451 ts_manager.cc:194] Registered new tserver with Master: b344c165f1164c868f060c933f4e151b (127.21.202.67:40793)
I20260806 19:45:23.125732 2451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:42861
I20260806 19:45:23.133596 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:45:23.136283 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 2533
Aug 06 19:45:23 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:45:23.164618 2968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:23.138474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44928 (local address 127.21.202.65:37031)
0806 19:45:23.139449 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:23.139456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:23.139485 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:45:23.139548 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:23.139554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:23.139807 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:23.139971 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:23.142234 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:23.143651 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:23.145669 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:23.146306 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:23.146438 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:23.152475 (+ 6037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:23.152494 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:23.152512 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:23.152559 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:23.157702 (+ 5143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:23.160507 (+ 2805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:23.160518 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:23.160520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:23.160756 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:23.161866 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:23.161872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:23.161876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:23.162304 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:23.162595 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:23.163089 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 19:45:23.163336 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":805,"thread_start_us":608,"threads_started":1}
I20260806 19:45:23.166383 2668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:23.272513 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 2533
I20260806 19:45:23.292413 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 2679
Aug 06 19:45:23 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:45:23.324674 2970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:23.294936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.21.202.66:45627)
0806 19:45:23.295991 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:23.296006 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:23.296046 (+ 40us) server_negotiation.cc:407] Connection header received
0806 19:45:23.296133 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:23.296142 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:23.296446 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:23.296645 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:23.300464 (+ 3819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:23.301675 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:23.303520 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:23.304445 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:23.304601 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:23.316137 (+ 11536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:23.316154 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:23.316171 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:23.316217 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:23.320644 (+ 4427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:23.321859 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:23.321869 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:23.321872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:23.322089 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:23.322688 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:23.322693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:23.322695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:23.323151 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:23.323539 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:23.323969 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 19:45:23.324359 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":601,"threads_started":1}
I20260806 19:45:23.326356 2820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:23.416110 2825 heartbeater.cc:503] Master 127.21.202.126:42035 was elected leader, sending a full tablet report...
I20260806 19:45:23.418025 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 2679
W20260806 19:45:23.441586 2437 connection.cc:535] server connection from 127.21.202.66:52603 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 19:45:23.442060 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 2831
Aug 06 19:45:23 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:45:23.466606 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:23.445177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51610 (local address 127.21.202.67:40793)
0806 19:45:23.446114 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:23.446121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:23.446152 (+ 31us) server_negotiation.cc:407] Connection header received
0806 19:45:23.446214 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:23.446220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:23.446447 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:23.446599 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:23.448209 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:23.449534 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:23.451114 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:23.451870 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:23.451997 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:23.458264 (+ 6267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:23.458287 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:23.458305 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:23.458406 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:23.462218 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:23.463181 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:23.463189 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:23.463192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:23.463372 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:23.464264 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:23.464269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:23.464271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:23.464688 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:23.465120 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:23.465743 (+ 623us) server_negotiation.cc:299] Negotiation successful
0806 19:45:23.466110 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":547,"threads_started":1}
I20260806 19:45:23.468258 2957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:23.558835 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 2831
I20260806 19:45:23.579468 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 2418
I20260806 19:45:23.581048 2483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:23.698230 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 2418
I20260806 19:45:23.729066 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42035
--webserver_interface=127.21.202.126
--webserver_port=45549
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:42035
--ranger_config_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.202.84:41071/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.202.84:41071/kms with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:45:24.145622 2975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:24.145977 2975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:24.146076 2975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:24.155616 2975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:45:24.155784 2975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:24.155834 2975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:24.155884 2975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:45:24.155932 2975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:45:24.170091 2975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.202.84:41071/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:42035
--ranger_config_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=45549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:24.172705 2975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:24.174487 2975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:24.188673 2985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:24.188601 2982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:24.188635 2981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:24.190384 2975 server_base.cc:1034] running on GCE node
I20260806 19:45:24.191351 2975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:24.193804 2975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:24.195240 2975 hybrid_clock.cc:648] HybridClock initialized: now 1786045524195167 us; error 70 us; skew 500 ppm
I20260806 19:45:24.195669 2975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:24 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045524, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:24.199528 2975 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:45:24.201447 2975 webserver.cc:460] Webserver started at http://127.21.202.126:45549/ using document root <none> and password file <none>
I20260806 19:45:24.202181 2975 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:24.202340 2975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 19:45:24 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045524, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:45:24.267637 2975 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 19:45:24.271375 2993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:24.273394 2975 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:45:24.273679 2975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "4612ddd609204b53ac0f93ff95a8bc5d"
format_stamp: "Formatted at 2026-08-06 19:45:19 on dist-test-slave-77v9"
server_key: "4357937988bcaaea731c7828bbea8363"
server_key_iv: "78ac967efc781f15756548251faf2edc"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:24.274744 2975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:24.295187 2975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:24.299775 2975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:24.300120 2975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:24.322144 2975 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:42035
I20260806 19:45:24.322185 3044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:42035 every 8 connection(s)
I20260806 19:45:24.323770 2975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:45:24.328677 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 2975
I20260806 19:45:24.329620 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.202.84:41071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:37031
--local_ip_for_outbound_sockets=127.21.202.65
--tserver_master_addrs=127.21.202.126:42035
--webserver_port=33551
--webserver_interface=127.21.202.65
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.202.84:41071/kms with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:45:24.331784 3045 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:45:24.335342 3045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d: Bootstrap starting.
I20260806 19:45:24.363372 3045 log.cc:824] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:24.378337 3045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:45:24.379315 3045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d: Bootstrap complete.
I20260806 19:45:24.394284 3045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42035 } }
I20260806 19:45:24.395735 3045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4612ddd609204b53ac0f93ff95a8bc5d, State: Initialized, Role: FOLLOWER
I20260806 19:45:24.396618 3045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42035 } }
I20260806 19:45:24.397100 3045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:45:24.397277 3045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:45:24.397545 3045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 1 FOLLOWER]: Advancing to term 2
I20260806 19:45:24.403852 3045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42035 } }
I20260806 19:45:24.405017 3045 leader_election.cc:302] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4612ddd609204b53ac0f93ff95a8bc5d; no voters:
I20260806 19:45:24.405714 3045 leader_election.cc:288] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 19:45:24.405910 3049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 2 FOLLOWER]: Leader election won for term 2
I20260806 19:45:24.407889 3045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:45:24.413340 3049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [term 2 LEADER]: Becoming Leader. State: Replica: 4612ddd609204b53ac0f93ff95a8bc5d, State: Running, Role: LEADER
I20260806 19:45:24.414278 3049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42035 } }
I20260806 19:45:24.418267 3051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4612ddd609204b53ac0f93ff95a8bc5d. Latest consensus state: current_term: 2 leader_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42035 } } }
I20260806 19:45:24.418699 3051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:24.417475 3050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612ddd609204b53ac0f93ff95a8bc5d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42035 } } }
I20260806 19:45:24.419345 3050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:24.427155 3064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:45:24.427608 3045 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 19:45:24.445858 3064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:45:24.448311 3064 catalog_manager.cc:1259] Loaded cluster ID: 89326ad3e2774d02abeacd1a37200bff
I20260806 19:45:24.448545 3064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:45:24.454105 3064 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:45:24.457034 3064 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d: Loaded TSK: 0
I20260806 19:45:24.459576 3064 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 19:45:24.880029 3047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:24.880548 3047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:24.880744 3047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:24.893363 3047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:24.893586 3047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:24.894301 3047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:45:24.912421 3047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.202.84:41071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:37031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=33551
--tserver_master_addrs=127.21.202.126:42035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:24.915561 3047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:24.917879 3047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:24.951468 3083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:24.952648 3084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:24.965691 3047 server_base.cc:1034] running on GCE node
W20260806 19:45:24.967308 3086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:24.968320 3047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:24.969863 3047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:24.971216 3047 hybrid_clock.cc:648] HybridClock initialized: now 1786045524971108 us; error 98 us; skew 500 ppm
I20260806 19:45:24.971668 3047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:24 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045524, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:24.976747 3047 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:45:24.979473 3047 webserver.cc:460] Webserver started at http://127.21.202.65:33551/ using document root <none> and password file <none>
I20260806 19:45:24.980540 3047 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:24.980767 3047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 19:45:25 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045524, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:45:25.056933 3047 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.007s sys 0.001s
I20260806 19:45:25.061136 3094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:25.063364 3047 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:45:25.063599 3047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "7d39b9875aa547e5b9d82f854e08c0f0"
format_stamp: "Formatted at 2026-08-06 19:45:21 on dist-test-slave-77v9"
server_key: "a9b2ce2c141c252941e7800c82e62002"
server_key_iv: "a490bf8c1602271f5f70ab392f663567"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:25.065295 3047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:25.091560 3047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:25.098238 3047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:25.098726 3047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:25.100324 3047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:25.102591 3047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:25.102757 3047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:25.102892 3047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:25.103020 3047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:25.211845 3047 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:37031
I20260806 19:45:25.211890 3212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:37031 every 8 connection(s)
I20260806 19:45:25.214875 3047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:45:25.221685 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 3047
I20260806 19:45:25.223109 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.202.84:41071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:45627
--local_ip_for_outbound_sockets=127.21.202.66
--tserver_master_addrs=127.21.202.126:42035
--webserver_port=40569
--webserver_interface=127.21.202.66
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.202.84:41071/kms with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:45:25 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045524, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:25.271521 3216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:25.218998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42035 (local address 127.21.202.65:43115)
0806 19:45:25.228730 (+ 9732us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:25.228874 (+ 144us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:25.236380 (+ 7506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:25.240759 (+ 4379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:25.240772 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:25.242170 (+ 1398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:25.243083 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:25.243122 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:25.247111 (+ 3989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:25.247127 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:25.249792 (+ 2665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:25.249821 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:25.252895 (+ 3074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:25.252908 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:25.253265 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:25.254491 (+ 1226us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:25.254529 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:25.258111 (+ 3582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:25.265043 (+ 6932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:25.265059 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:25.265092 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:25.266168 (+ 1076us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:25.266990 (+ 822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:25.266996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:25.267000 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:25.267296 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:25.268607 (+ 1311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:25.268616 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:25.269088 (+ 472us) client_negotiation.cc:769] Sending connection context
0806 19:45:25.269400 (+ 312us) client_negotiation.cc:240] Negotiation successful
0806 19:45:25.270031 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9170,"thread_start_us":12627,"threads_started":1}
I20260806 19:45:25.273483 3215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:25.218793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:43115 (local address 127.21.202.126:42035)
0806 19:45:25.238546 (+ 19753us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:25.238566 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:25.238635 (+ 69us) server_negotiation.cc:407] Connection header received
0806 19:45:25.239256 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:25.239270 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:25.239727 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:25.240306 (+ 579us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:25.243430 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:25.246744 (+ 3314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:25.250104 (+ 3360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:25.252088 (+ 1984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:25.252369 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:25.258638 (+ 6269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:25.258653 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:25.258680 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:25.258760 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:25.264664 (+ 5904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:25.266439 (+ 1775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:25.266476 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:25.266488 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:25.266778 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:25.267488 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:25.267494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:25.267497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:25.268306 (+ 809us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:25.268525 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:25.270688 (+ 2163us) server_negotiation.cc:299] Negotiation successful
0806 19:45:25.271268 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19444,"thread_start_us":19158,"threads_started":1}
I20260806 19:45:25.279258 3213 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42035
I20260806 19:45:25.279644 3213 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:25.280786 3213 heartbeater.cc:511] Master 127.21.202.126:42035 requested a full tablet report, sending...
W20260806 19:45:25.284536 3213 heartbeater.cc:644] Failed to heartbeat to 127.21.202.126:42035 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 19:45:25.284898 3213 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:25.285295 3213 heartbeater.cc:511] Master 127.21.202.126:42035 requested a full tablet report, sending...
I20260806 19:45:25.287215 3213 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:25.287698 3213 heartbeater.cc:511] Master 127.21.202.126:42035 requested a full tablet report, sending...
W20260806 19:45:25.290594 3213 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 19:45:25.763490 3217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:25.763979 3217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:25.764159 3217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:25.773312 3217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:25.773458 3217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:25.773689 3217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:45:25.791054 3217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.202.84:41071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:45627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=40569
--tserver_master_addrs=127.21.202.126:42035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:25.794024 3217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:25.796166 3217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:25.838482 3234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:25.839526 3235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:25.844973 3217 server_base.cc:1034] running on GCE node
W20260806 19:45:25.846298 3237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:25.847425 3217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:25.854641 3217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:25.858170 3217 hybrid_clock.cc:648] HybridClock initialized: now 1786045525858047 us; error 110 us; skew 500 ppm
I20260806 19:45:25.858714 3217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:25 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045525, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:25.863042 3217 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:45:25.866681 3217 webserver.cc:460] Webserver started at http://127.21.202.66:40569/ using document root <none> and password file <none>
I20260806 19:45:25.867805 3217 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:25.868124 3217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 19:45:25 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045525, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:45:25.899317 3217 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 19:45:25.904753 3246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:25.906881 3217 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:45:25.907155 3217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "5bc4e48af34842328e525c271e7a575e"
format_stamp: "Formatted at 2026-08-06 19:45:22 on dist-test-slave-77v9"
server_key: "1ded3ef5301a09406554b9c8142246df"
server_key_iv: "912d27882437c455fcd3988123782994"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:25.908586 3217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 06 19:45:25 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786045525, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:25.971735 3217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:25.980103 3217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:25.980790 3217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:25.991272 3217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:25.995366 3217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:25.995723 3217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 19:45:25.995942 3217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:25.996088 3217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:26.066859 3217 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:45627
I20260806 19:45:26.066910 3360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:45627 every 8 connection(s)
I20260806 19:45:26.069506 3217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:45:26.080240 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 3217
I20260806 19:45:26.081442 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.202.84:41071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:40793
--local_ip_for_outbound_sockets=127.21.202.67
--tserver_master_addrs=127.21.202.126:42035
--webserver_port=40843
--webserver_interface=127.21.202.67
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.202.84:41071/kms with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:45:26 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045525, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:26.155879 3363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:26.075081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42035 (local address 127.21.202.66:41105)
0806 19:45:26.111265 (+ 36184us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:26.111440 (+ 175us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:26.127292 (+ 15852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:26.129538 (+ 2246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:26.129560 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:26.130414 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:26.131521 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:26.131559 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:26.133494 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:26.133504 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:26.136123 (+ 2619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:26.136148 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:26.137524 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:26.137536 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:26.137983 (+ 447us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:26.138988 (+ 1005us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:26.139037 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:26.142472 (+ 3435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:26.149395 (+ 6923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:26.149423 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:26.149448 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:26.150257 (+ 809us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:26.150886 (+ 629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:26.150893 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:26.150897 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:26.151201 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:26.152758 (+ 1557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:26.152769 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:26.153375 (+ 606us) client_negotiation.cc:769] Sending connection context
0806 19:45:26.153767 (+ 392us) client_negotiation.cc:240] Negotiation successful
0806 19:45:26.154384 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35601,"thread_start_us":39470,"threads_started":1}
I20260806 19:45:26.157490 3364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:26.076019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:41105 (local address 127.21.202.126:42035)
0806 19:45:26.110128 (+ 34109us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:26.110139 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:26.112230 (+ 2091us) server_negotiation.cc:407] Connection header received
0806 19:45:26.128039 (+ 15809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:26.128052 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:26.128404 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:26.128600 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:26.131878 (+ 3278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:26.133231 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:26.136379 (+ 3148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:26.137179 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:26.137340 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:26.143399 (+ 6059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:26.143409 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:26.143416 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:26.143471 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:26.149101 (+ 5630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:26.150502 (+ 1401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:26.150509 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:26.150513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:26.150706 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:26.151377 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:26.151384 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:26.151387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:26.151835 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:26.152620 (+ 785us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:26.156631 (+ 4011us) server_negotiation.cc:299] Negotiation successful
0806 19:45:26.157048 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33866,"thread_start_us":34139,"threads_started":1}
I20260806 19:45:26.161928 3361 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42035
I20260806 19:45:26.162614 3361 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:26.164268 3361 heartbeater.cc:511] Master 127.21.202.126:42035 requested a full tablet report, sending...
I20260806 19:45:26.167502 3010 ts_manager.cc:194] Registered new tserver with Master: 5bc4e48af34842328e525c271e7a575e (127.21.202.66:45627)
I20260806 19:45:26.170954 3010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:41105
I20260806 19:45:26.290756 3213 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:26.291301 3213 heartbeater.cc:511] Master 127.21.202.126:42035 requested a full tablet report, sending...
I20260806 19:45:26.292913 3010 ts_manager.cc:194] Registered new tserver with Master: 7d39b9875aa547e5b9d82f854e08c0f0 (127.21.202.65:37031)
I20260806 19:45:26.294253 3010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:43115
Aug 06 19:45:26 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045525, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
W20260806 19:45:26.694826 3365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:26.695297 3365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:26.695530 3365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:26.709548 3365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:26.709760 3365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:26.709983 3365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:45:26.727030 3365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.202.84:41071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:40793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=40843
--tserver_master_addrs=127.21.202.126:42035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:26.729184 3365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:26.731166 3365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:26.754678 3371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:26.761770 3370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:26.767262 3373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:26.767778 3365 server_base.cc:1034] running on GCE node
I20260806 19:45:26.768565 3365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:26.769894 3365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:26.771382 3365 hybrid_clock.cc:648] HybridClock initialized: now 1786045526771258 us; error 64 us; skew 500 ppm
I20260806 19:45:26.772735 3365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:26 dist-test-slave-77v9 krb5kdc[1958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045526, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:26.779608 3365 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:45:26.790861 3365 webserver.cc:460] Webserver started at http://127.21.202.67:40843/ using document root <none> and password file <none>
I20260806 19:45:26.791908 3365 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:26.792140 3365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 19:45:26 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045526, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:45:26.823328 3365 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.006s sys 0.000s
I20260806 19:45:26.833658 3382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:26.836153 3365 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.003s
I20260806 19:45:26.836540 3365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "b344c165f1164c868f060c933f4e151b"
format_stamp: "Formatted at 2026-08-06 19:45:22 on dist-test-slave-77v9"
server_key: "e9f076dc4395ae206f490a3601b4f8b8"
server_key_iv: "10c023be41d609fe1d5bdbeb8b6e8834"
server_key_version: "kuduclusterkey@0"
I20260806 19:45:26.838251 3365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 06 19:45:26 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045525, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for HTTP/127.21.202.84@KRBTEST.COM
I20260806 19:45:26.928999 3365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:26.936466 3365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:26.936978 3365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:26.953423 3365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:26.967258 3365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:26.967895 3365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260806 19:45:26.968173 3365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:26.968310 3365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:27.163810 3365 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:40793
I20260806 19:45:27.163859 3500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:40793 every 8 connection(s)
I20260806 19:45:27.166524 3365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:45:27.173834 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 3365
I20260806 19:45:27.177093 3361 heartbeater.cc:503] Master 127.21.202.126:42035 was elected leader, sending a full tablet report...
Aug 06 19:45:27 dist-test-slave-77v9 krb5kdc[1958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045526, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:27.223096 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.176151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.21.202.126:42035)
0806 19:45:27.181509 (+ 5358us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:27.181516 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:27.181539 (+ 23us) server_negotiation.cc:407] Connection header received
0806 19:45:27.181607 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:27.181614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:27.181732 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:27.184900 (+ 3168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:27.185165 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.186746 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.192044 (+ 5298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.193156 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.193347 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.202793 (+ 9446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:27.202801 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:27.202807 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:27.202874 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:27.207680 (+ 4806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:27.209038 (+ 1358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:27.209047 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:27.209052 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:27.209286 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:27.209795 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:27.209800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:27.209804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:27.210203 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:27.210481 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:27.216688 (+ 6207us) server_negotiation.cc:299] Negotiation successful
0806 19:45:27.222593 (+ 5905us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5228,"thread_start_us":5015,"threads_started":1}
I20260806 19:45:27.227083 3503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.172320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42035 (local address 127.21.202.67:33129)
0806 19:45:27.190685 (+ 18365us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:27.190762 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:27.193621 (+ 2859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:27.195118 (+ 1497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:27.195136 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:27.195937 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:27.197310 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:27.197349 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.200199 (+ 2850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.200207 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:27.202964 (+ 2757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:27.202995 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.204797 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.204833 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:27.205191 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.206499 (+ 1308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:27.206564 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:27.212236 (+ 5672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:27.218839 (+ 6603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:27.218856 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:27.218887 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:27.220267 (+ 1380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:27.221047 (+ 780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:27.221053 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:27.221057 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:27.221394 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:27.222969 (+ 1575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:27.223087 (+ 118us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:27.223773 (+ 686us) client_negotiation.cc:769] Sending connection context
0806 19:45:27.224281 (+ 508us) client_negotiation.cc:240] Negotiation successful
0806 19:45:27.225076 (+ 795us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17900,"thread_start_us":18419,"threads_started":1}
I20260806 19:45:27.228567 3504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.174837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:33129 (local address 127.21.202.126:42035)
0806 19:45:27.175439 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:27.175447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:27.190902 (+ 15455us) server_negotiation.cc:407] Connection header received
0806 19:45:27.194053 (+ 3151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:27.194059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:27.194185 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:27.194300 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:27.198177 (+ 3877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.199744 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.203234 (+ 3490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.204185 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.204355 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.212600 (+ 8245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:27.212610 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:27.212615 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:27.212669 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:27.218435 (+ 5766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:27.220566 (+ 2131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:27.220574 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:27.220578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:27.220868 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:27.221573 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:27.221579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:27.221582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:27.222378 (+ 796us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:27.222802 (+ 424us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:27.227764 (+ 4962us) server_negotiation.cc:299] Negotiation successful
0806 19:45:27.228144 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":1418,"threads_started":1}
I20260806 19:45:27.231094 3501 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42035
I20260806 19:45:27.231468 3501 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:27.232511 3501 heartbeater.cc:511] Master 127.21.202.126:42035 requested a full tablet report, sending...
I20260806 19:45:27.234660 3009 ts_manager.cc:194] Registered new tserver with Master: b344c165f1164c868f060c933f4e151b (127.21.202.67:40793)
I20260806 19:45:27.236028 3009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:33129
I20260806 19:45:27.244511 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:45:27.269896 3504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.249639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60642 (local address 127.21.202.126:42035)
0806 19:45:27.249851 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:27.249858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:27.249885 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:45:27.250045 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:27.250051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:27.250142 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:27.250287 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:27.253218 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.254456 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.256135 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.256976 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.257166 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.261587 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:27.261596 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:27.261601 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:27.261659 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:27.265420 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:27.266809 (+ 1389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:27.266818 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:27.266822 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:27.267031 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:27.267663 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:27.267669 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:27.267673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:27.268124 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:27.268498 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:27.269139 (+ 641us) server_negotiation.cc:299] Negotiation successful
0806 19:45:27.269442 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 19:45:27.278059 3009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:45:27.288678 3009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:45:27.298476 3213 heartbeater.cc:503] Master 127.21.202.126:42035 was elected leader, sending a full tablet report...
I20260806 19:45:27.359884 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.314771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38788 (local address 127.21.202.66:45627)
0806 19:45:27.339348 (+ 24577us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:27.339359 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:27.340478 (+ 1119us) server_negotiation.cc:407] Connection header received
0806 19:45:27.340598 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:27.340607 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:27.341018 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:27.341305 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:27.343451 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.347625 (+ 4174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.352281 (+ 4656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.354862 (+ 2581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.355070 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.355381 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:27.358689 (+ 3308us) server_negotiation.cc:299] Negotiation successful
0806 19:45:27.359225 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24330,"thread_start_us":24600,"threads_started":1}
I20260806 19:45:27.360781 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.313762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:45627 (local address 127.0.0.1:38788)
0806 19:45:27.340116 (+ 26354us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:27.340140 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:27.340329 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:27.341648 (+ 1319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:27.341662 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:27.341717 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:27.342147 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:27.342163 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.347847 (+ 5684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.347853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:27.351867 (+ 4014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:27.351905 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.355639 (+ 3734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.355650 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:27.355928 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.356174 (+ 246us) client_negotiation.cc:769] Sending connection context
0806 19:45:27.356382 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 19:45:27.356803 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26205,"mutex_wait_us":1,"thread_start_us":31507,"threads_started":1}
I20260806 19:45:27.368024 3525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.311749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37031 (local address 127.0.0.1:44936)
0806 19:45:27.312605 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:27.312659 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:27.312870 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:27.342262 (+ 29392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:27.342266 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:27.342293 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:27.342711 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:27.342728 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.358353 (+ 15625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.358363 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:27.364005 (+ 5642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:27.364034 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.366748 (+ 2714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.366754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:27.366961 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.367075 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 19:45:27.367180 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 19:45:27.367525 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":52,"thread_start_us":382,"threads_started":1}
I20260806 19:45:27.368321 3295 tablet_service.cc:1467] Processing CreateTablet for tablet 07627b8cf967425388b29e67a530a9a1 (DEFAULT_TABLE table=test-table [id=9c660a5427e642d99636a6080f0f647f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:45:27.369799 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.312179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44936 (local address 127.21.202.65:37031)
0806 19:45:27.341525 (+ 29346us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:27.341535 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:27.341563 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:45:27.341662 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:27.341670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:27.342057 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:27.342283 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:27.346081 (+ 3798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.352063 (+ 5982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.364402 (+ 12339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.366419 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.368724 (+ 2305us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.369122 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:27.369223 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 19:45:27.369439 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29104,"thread_start_us":10408,"threads_started":1}
I20260806 19:45:27.373131 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.314196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:40793 (local address 127.0.0.1:51614)
0806 19:45:27.343735 (+ 29539us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:27.343774 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:27.344027 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:27.348500 (+ 4473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:27.348506 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:27.348537 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:27.352800 (+ 4263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:27.352859 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.366433 (+ 13574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.366440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:27.368952 (+ 2512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:27.368970 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.371997 (+ 3027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.372007 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:27.372372 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.372414 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 19:45:27.372538 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 19:45:27.372700 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29236,"mutex_wait_us":1063,"thread_start_us":29959,"threads_started":1}
I20260806 19:45:27.374205 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.314637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51614 (local address 127.21.202.67:40793)
0806 19:45:27.347456 (+ 32819us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:27.347465 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:27.347502 (+ 37us) server_negotiation.cc:407] Connection header received
0806 19:45:27.347612 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:27.347620 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:27.347944 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:27.348175 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:27.353280 (+ 5105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.366089 (+ 12809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.369235 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.371266 (+ 2031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.373388 (+ 2122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.373696 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:27.373766 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 19:45:27.373924 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32619,"thread_start_us":32452,"threads_started":1}
I20260806 19:45:27.375201 3146 tablet_service.cc:1467] Processing CreateTablet for tablet 07627b8cf967425388b29e67a530a9a1 (DEFAULT_TABLE table=test-table [id=9c660a5427e642d99636a6080f0f647f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:45:27.377171 3146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07627b8cf967425388b29e67a530a9a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:27.377202 3295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07627b8cf967425388b29e67a530a9a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:27.380941 3434 tablet_service.cc:1467] Processing CreateTablet for tablet 07627b8cf967425388b29e67a530a9a1 (DEFAULT_TABLE table=test-table [id=9c660a5427e642d99636a6080f0f647f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:45:27.382933 3434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07627b8cf967425388b29e67a530a9a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:27.400801 3533 tablet_bootstrap.cc:492] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0: Bootstrap starting.
I20260806 19:45:27.409439 3532 tablet_bootstrap.cc:492] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e: Bootstrap starting.
I20260806 19:45:27.414590 3532 tablet_bootstrap.cc:654] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:27.415343 3533 tablet_bootstrap.cc:654] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:27.416388 3532 log.cc:824] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:27.417342 3533 log.cc:824] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:27.418857 3534 tablet_bootstrap.cc:492] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b: Bootstrap starting.
I20260806 19:45:27.422978 3533 tablet_bootstrap.cc:492] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0: No bootstrap required, opened a new log
I20260806 19:45:27.423358 3533 ts_tablet_manager.cc:1397] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.008s
I20260806 19:45:27.425166 3534 tablet_bootstrap.cc:654] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:27.427316 3534 log.cc:824] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:27.427866 3532 tablet_bootstrap.cc:492] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e: No bootstrap required, opened a new log
I20260806 19:45:27.428270 3532 ts_tablet_manager.cc:1397] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.000s
I20260806 19:45:27.439791 3534 tablet_bootstrap.cc:492] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b: No bootstrap required, opened a new log
I20260806 19:45:27.440325 3534 ts_tablet_manager.cc:1397] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.008s
I20260806 19:45:27.442368 3532 raft_consensus.cc:348] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37031 } } peers { permanent_uuid: "5bc4e48af34842328e525c271e7a575e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45627 } } peers { permanent_uuid: "b344c165f1164c868f060c933f4e151b" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40793 } }
I20260806 19:45:27.443137 3532 raft_consensus.cc:374] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:27.443295 3532 raft_consensus.cc:729] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bc4e48af34842328e525c271e7a575e, State: Initialized, Role: FOLLOWER
I20260806 19:45:27.444293 3532 consensus_queue.cc:260] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37031 } } peers { permanent_uuid: "5bc4e48af34842328e525c271e7a575e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45627 } } peers { permanent_uuid: "b344c165f1164c868f060c933f4e151b" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40793 } }
I20260806 19:45:27.449581 3533 raft_consensus.cc:348] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37031 } } peers { permanent_uuid: "5bc4e48af34842328e525c271e7a575e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45627 } } peers { permanent_uuid: "b344c165f1164c868f060c933f4e151b" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40793 } }
I20260806 19:45:27.450321 3533 raft_consensus.cc:374] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:27.451956 3533 raft_consensus.cc:729] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d39b9875aa547e5b9d82f854e08c0f0, State: Initialized, Role: FOLLOWER
I20260806 19:45:27.453497 3533 consensus_queue.cc:260] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37031 } } peers { permanent_uuid: "5bc4e48af34842328e525c271e7a575e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45627 } } peers { permanent_uuid: "b344c165f1164c868f060c933f4e151b" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40793 } }
I20260806 19:45:27.454263 3534 raft_consensus.cc:348] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37031 } } peers { permanent_uuid: "5bc4e48af34842328e525c271e7a575e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45627 } } peers { permanent_uuid: "b344c165f1164c868f060c933f4e151b" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40793 } }
I20260806 19:45:27.454876 3534 raft_consensus.cc:374] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:27.455020 3534 raft_consensus.cc:729] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b344c165f1164c868f060c933f4e151b, State: Initialized, Role: FOLLOWER
I20260806 19:45:27.455843 3534 consensus_queue.cc:260] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37031 } } peers { permanent_uuid: "5bc4e48af34842328e525c271e7a575e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45627 } } peers { permanent_uuid: "b344c165f1164c868f060c933f4e151b" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40793 } }
I20260806 19:45:27.456300 3532 ts_tablet_manager.cc:1428] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e: Time spent starting tablet: real 0.028s user 0.019s sys 0.004s
I20260806 19:45:27.459554 3533 ts_tablet_manager.cc:1428] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0: Time spent starting tablet: real 0.036s user 0.016s sys 0.004s
I20260806 19:45:27.461757 3501 heartbeater.cc:503] Master 127.21.202.126:42035 was elected leader, sending a full tablet report...
I20260806 19:45:27.462869 3534 ts_tablet_manager.cc:1428] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b: Time spent starting tablet: real 0.022s user 0.014s sys 0.003s
W20260806 19:45:27.493122 3214 tablet.cc:2369] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:45:27.582341 3540 raft_consensus.cc:482] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:45:27.582998 3540 raft_consensus.cc:504] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37031 } } peers { permanent_uuid: "5bc4e48af34842328e525c271e7a575e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45627 } } peers { permanent_uuid: "b344c165f1164c868f060c933f4e151b" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40793 } }
I20260806 19:45:27.586148 3540 leader_election.cc:288] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bc4e48af34842328e525c271e7a575e (127.21.202.66:45627), b344c165f1164c868f060c933f4e151b (127.21.202.67:40793)
I20260806 19:45:27.594877 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.587092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:40205 (local address 127.21.202.67:40793)
0806 19:45:27.587508 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:27.587516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:27.588465 (+ 949us) server_negotiation.cc:407] Connection header received
0806 19:45:27.588665 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:27.588669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:27.588729 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:27.588855 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:27.589467 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.590341 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.593015 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.593815 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.594049 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.594131 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:27.594449 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 19:45:27.594609 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260806 19:45:27.595090 3543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.586945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:40793 (local address 127.21.202.65:40205)
0806 19:45:27.588341 (+ 1396us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:27.588356 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:27.588506 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:27.588888 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:27.588892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:27.588930 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:27.589294 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:27.589303 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.590543 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.590548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:27.592771 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:27.592787 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.594067 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.594074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:27.594266 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.594293 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 19:45:27.594415 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 19:45:27.594570 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1257,"mutex_wait_us":280,"thread_start_us":1039,"threads_started":1}
I20260806 19:45:27.596403 3455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07627b8cf967425388b29e67a530a9a1" candidate_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b344c165f1164c868f060c933f4e151b" is_pre_election: true
I20260806 19:45:27.596709 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.587588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:42877 (local address 127.21.202.66:45627)
0806 19:45:27.587805 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:27.587811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:27.587835 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:45:27.588046 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:27.588052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:27.588137 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:27.588323 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:27.589340 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.590793 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.593942 (+ 3149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.595313 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.595466 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.595542 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:27.596273 (+ 731us) server_negotiation.cc:299] Negotiation successful
0806 19:45:27.596433 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 19:45:27.596834 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.586289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:45627 (local address 127.21.202.65:42877)
0806 19:45:27.587419 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:27.587442 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:27.587675 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:27.588528 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:27.588534 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:27.588566 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:27.589093 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:27.589114 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.591004 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.591010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:27.593700 (+ 2690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:27.593721 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.595772 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.595781 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:27.596050 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.596091 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 19:45:27.596217 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 19:45:27.596372 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"mutex_wait_us":3,"thread_start_us":847,"threads_started":1}
I20260806 19:45:27.597280 3455 raft_consensus.cc:2454] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d39b9875aa547e5b9d82f854e08c0f0 in term 0.
I20260806 19:45:27.598165 3315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07627b8cf967425388b29e67a530a9a1" candidate_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bc4e48af34842328e525c271e7a575e" is_pre_election: true
I20260806 19:45:27.598457 3097 leader_election.cc:302] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d39b9875aa547e5b9d82f854e08c0f0, b344c165f1164c868f060c933f4e151b; no voters:
I20260806 19:45:27.598827 3315 raft_consensus.cc:2454] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d39b9875aa547e5b9d82f854e08c0f0 in term 0.
I20260806 19:45:27.599077 3540 raft_consensus.cc:2781] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:45:27.599231 3540 raft_consensus.cc:482] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:45:27.599339 3540 raft_consensus.cc:3037] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:27.602895 3540 raft_consensus.cc:504] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37031 } } peers { permanent_uuid: "5bc4e48af34842328e525c271e7a575e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45627 } } peers { permanent_uuid: "b344c165f1164c868f060c933f4e151b" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40793 } }
I20260806 19:45:27.604498 3540 leader_election.cc:288] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [CANDIDATE]: Term 1 election: Requested vote from peers 5bc4e48af34842328e525c271e7a575e (127.21.202.66:45627), b344c165f1164c868f060c933f4e151b (127.21.202.67:40793)
I20260806 19:45:27.605177 3315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07627b8cf967425388b29e67a530a9a1" candidate_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bc4e48af34842328e525c271e7a575e"
I20260806 19:45:27.605181 3455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07627b8cf967425388b29e67a530a9a1" candidate_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b344c165f1164c868f060c933f4e151b"
I20260806 19:45:27.605466 3455 raft_consensus.cc:3037] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:27.605460 3315 raft_consensus.cc:3037] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:27.609915 3315 raft_consensus.cc:2454] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d39b9875aa547e5b9d82f854e08c0f0 in term 1.
I20260806 19:45:27.609915 3455 raft_consensus.cc:2454] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d39b9875aa547e5b9d82f854e08c0f0 in term 1.
I20260806 19:45:27.610718 3098 leader_election.cc:302] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bc4e48af34842328e525c271e7a575e, 7d39b9875aa547e5b9d82f854e08c0f0; no voters:
I20260806 19:45:27.611485 3540 raft_consensus.cc:2781] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:45:27.612268 3540 raft_consensus.cc:686] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [term 1 LEADER]: Becoming Leader. State: Replica: 7d39b9875aa547e5b9d82f854e08c0f0, State: Running, Role: LEADER
I20260806 19:45:27.613483 3540 consensus_queue.cc:237] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37031 } } peers { permanent_uuid: "5bc4e48af34842328e525c271e7a575e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45627 } } peers { permanent_uuid: "b344c165f1164c868f060c933f4e151b" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40793 } }
I20260806 19:45:27.621169 3008 catalog_manager.cc:5495] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d39b9875aa547e5b9d82f854e08c0f0 (127.21.202.65). New cstate: current_term: 1 leader_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d39b9875aa547e5b9d82f854e08c0f0" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bc4e48af34842328e525c271e7a575e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b344c165f1164c868f060c933f4e151b" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40793 } health_report { overall_health: UNKNOWN } } }
W20260806 19:45:27.625479 3362 tablet.cc:2369] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:45:27.673983 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.663437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44950 (local address 127.21.202.65:37031)
0806 19:45:27.663890 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:27.663899 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:27.664259 (+ 360us) server_negotiation.cc:407] Connection header received
0806 19:45:27.664632 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:27.664643 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:27.664754 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:27.664987 (+ 233us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:45:27.665692 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.666990 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.669611 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.670371 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.670536 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.671550 (+ 1014us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:45:27.672386 (+ 836us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:45:27.672565 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:27.673318 (+ 753us) server_negotiation.cc:299] Negotiation successful
0806 19:45:27.673562 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260806 19:45:27.675055 3502 tablet.cc:2369] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:45:27.686215 3455 raft_consensus.cc:1261] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b [term 1 FOLLOWER]: Refusing update from remote peer 7d39b9875aa547e5b9d82f854e08c0f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:45:27.686218 3315 raft_consensus.cc:1261] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e [term 1 FOLLOWER]: Refusing update from remote peer 7d39b9875aa547e5b9d82f854e08c0f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:45:27.688349 3540 consensus_queue.cc:1035] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b344c165f1164c868f060c933f4e151b" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:45:27.688952 3544 consensus_queue.cc:1035] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bc4e48af34842328e525c271e7a575e" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:45:27.706507 3547 mvcc.cc:205] Tried to move back new op lower bound from 7315642481388322816 to 7315642481112363008. Current Snapshot: MvccSnapshot[applied={T|T < 7315642481388322816}]
I20260806 19:45:27.726698 3008 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:45:27.727035 3008 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:45:27.731679 3008 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4612ddd609204b53ac0f93ff95a8bc5d: Sending DeleteTablet for 3 replicas of tablet 07627b8cf967425388b29e67a530a9a1
I20260806 19:45:27.733794 3295 tablet_service.cc:1514] Processing DeleteTablet for tablet 07627b8cf967425388b29e67a530a9a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:45:27 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:38788
I20260806 19:45:27.733794 3146 tablet_service.cc:1514] Processing DeleteTablet for tablet 07627b8cf967425388b29e67a530a9a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:45:27 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:44936
I20260806 19:45:27.734074 3434 tablet_service.cc:1514] Processing DeleteTablet for tablet 07627b8cf967425388b29e67a530a9a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:45:27 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:51614
I20260806 19:45:27.734994 3556 tablet_replica.cc:330] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e: stopping tablet replica
I20260806 19:45:27.735728 3557 tablet_replica.cc:330] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0: stopping tablet replica
I20260806 19:45:27.736377 3556 raft_consensus.cc:2229] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:45:27.736369 3557 raft_consensus.cc:2229] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:45:27.737321 3556 raft_consensus.cc:2258] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:45:27.737567 3557 raft_consensus.cc:2258] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:45:27.740015 3556 ts_tablet_manager.cc:1905] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:45:27.741712 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 3047
I20260806 19:45:27.746662 3557 ts_tablet_manager.cc:1905] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:45:27.750403 3558 tablet_replica.cc:330] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b: stopping tablet replica
I20260806 19:45:27.751041 3558 raft_consensus.cc:2229] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:45:27.753747 3558 raft_consensus.cc:2258] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:45:27.754522 3556 ts_tablet_manager.cc:1918] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:45:27.754806 3556 log.cc:1196] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/07627b8cf967425388b29e67a530a9a1
I20260806 19:45:27.755757 3556 ts_tablet_manager.cc:1939] T 07627b8cf967425388b29e67a530a9a1 P 5bc4e48af34842328e525c271e7a575e: Deleting consensus metadata
I20260806 19:45:27.755752 3557 ts_tablet_manager.cc:1918] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:45:27.755936 3557 log.cc:1196] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/07627b8cf967425388b29e67a530a9a1
I20260806 19:45:27.756225 3558 ts_tablet_manager.cc:1905] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:45:27.757066 3557 ts_tablet_manager.cc:1939] T 07627b8cf967425388b29e67a530a9a1 P 7d39b9875aa547e5b9d82f854e08c0f0: Deleting consensus metadata
I20260806 19:45:27.758330 2996 catalog_manager.cc:4841] TS 5bc4e48af34842328e525c271e7a575e (127.21.202.66:45627): tablet 07627b8cf967425388b29e67a530a9a1 (table test-table [id=9c660a5427e642d99636a6080f0f647f]) successfully deleted
I20260806 19:45:27.759806 2994 catalog_manager.cc:4841] TS 7d39b9875aa547e5b9d82f854e08c0f0 (127.21.202.65:37031): tablet 07627b8cf967425388b29e67a530a9a1 (table test-table [id=9c660a5427e642d99636a6080f0f647f]) successfully deleted
I20260806 19:45:27.768579 3558 ts_tablet_manager.cc:1918] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:45:27.769366 3558 log.cc:1196] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/07627b8cf967425388b29e67a530a9a1
I20260806 19:45:27.770201 3558 ts_tablet_manager.cc:1939] T 07627b8cf967425388b29e67a530a9a1 P b344c165f1164c868f060c933f4e151b: Deleting consensus metadata
I20260806 19:45:27.772687 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.744977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44956 (local address 127.21.202.65:37031)
0806 19:45:27.745256 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:27.745267 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:27.749673 (+ 4406us) server_negotiation.cc:407] Connection header received
0806 19:45:27.749944 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:27.749952 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:27.750370 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:27.750498 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:27.753245 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.754525 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.759419 (+ 4894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.760133 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.760324 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.763448 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:27.763462 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:27.763479 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:27.763529 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:27.768119 (+ 4590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:27.770066 (+ 1947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:27.770075 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:27.770077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:27.770316 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:27.770812 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:27.770821 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:27.770823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:27.771320 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:27.771761 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:27.771945 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 19:45:27.772231 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 19:45:27.773313 2995 catalog_manager.cc:4841] TS b344c165f1164c868f060c933f4e151b (127.21.202.67:40793): tablet 07627b8cf967425388b29e67a530a9a1 (table test-table [id=9c660a5427e642d99636a6080f0f647f]) successfully deleted
I20260806 19:45:27.773784 3207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:27.887097 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 3047
I20260806 19:45:27.906519 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 3217
I20260806 19:45:27.923352 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:27.907675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38802 (local address 127.21.202.66:45627)
0806 19:45:27.907906 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:27.907915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:27.908323 (+ 408us) server_negotiation.cc:407] Connection header received
0806 19:45:27.908419 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:27.908423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:27.908516 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:27.908649 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:27.910268 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.911231 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.912799 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:27.913376 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:27.913543 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:27.916477 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:27.916493 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:27.916510 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:27.916556 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:27.920020 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:27.921056 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:27.921065 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:27.921067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:27.921226 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:27.921655 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:27.921659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:27.921661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:27.922121 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:27.922407 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:27.922697 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 19:45:27.922937 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 19:45:27.924196 3356 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:28.021322 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 3217
I20260806 19:45:28.041576 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 3365
I20260806 19:45:28.063685 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:28.042552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51620 (local address 127.21.202.67:40793)
0806 19:45:28.042739 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:28.042744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:28.042765 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:45:28.042874 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:28.042878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:28.042947 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:28.043040 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:28.044666 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:28.045903 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:28.047650 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:28.048158 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:28.048286 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:28.056708 (+ 8422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:28.056728 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:28.056747 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:28.056951 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:28.060139 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:28.061216 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:28.061224 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:28.061227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:28.061386 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:28.061828 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:28.061835 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:28.061838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:28.062222 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:28.062678 (+ 456us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:28.063066 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 19:45:28.063310 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 19:45:28.064491 3496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:28.166844 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 3365
I20260806 19:45:28.187896 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 2975
I20260806 19:45:28.189507 3040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:28.328295 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 2975
I20260806 19:45:28.354964 22313 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 19:45:28.703078 22313 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 19:45:28.703390 22313 mini_ranger.cc:66] Stopping Ranger...
I20260806 19:45:29.051900 22313 mini_ranger.cc:68] Stopped Ranger
2026-08-06 19:45:29.052 UTC [1989] LOG: received smart shutdown request
2026-08-06 19:45:29.057 UTC [1989] LOG: background worker "logical replication launcher" (PID 1998) exited with exit code 1
2026-08-06 19:45:29.058 UTC [1993] LOG: shutting down
2026-08-06 19:45:29.337 UTC [1989] LOG: database system is shut down
2026-08-06T19:45:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (60154 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:45:29 dist-test-slave-77v9 krb5kdc[3572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:45:29 dist-test-slave-77v9 krb5kdc[3572](info): set up 2 sockets
Aug 06 19:45:29 dist-test-slave-77v9 krb5kdc[3572](info): commencing operation
krb5kdc: starting...
W20260806 19:45:30.748315 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.012s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:45:30 dist-test-slave-77v9 krb5kdc[3572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:45:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:45:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:45:31.100303 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:38851
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:33813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:38851,127.21.202.125:33483,127.21.202.124:41257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:45:31.554373 3585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:31.554883 3585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:31.555059 3585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:31.567127 3585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:45:31.567294 3585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:31.567346 3585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:31.567487 3585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:45:31.567571 3585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:45:31.583115 3585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:33813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:38851,127.21.202.125:33483,127.21.202.124:41257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:38851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:31.584733 3585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:31.586496 3585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:31.602106 3590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:31.602152 3591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:31.602093 3593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:31.603349 3585 server_base.cc:1034] running on GCE node
I20260806 19:45:31.604442 3585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:31.606357 3585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:31.607923 3585 hybrid_clock.cc:648] HybridClock initialized: now 1786045531607736 us; error 113 us; skew 500 ppm
I20260806 19:45:31.608619 3585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:31 dist-test-slave-77v9 krb5kdc[3572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045531, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:31.613806 3585 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:45:31.615787 3585 webserver.cc:460] Webserver started at http://127.21.202.126:45477/ using document root <none> and password file <none>
I20260806 19:45:31.616626 3585 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:31.616796 3585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:31.617237 3585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:45:31.620977 3585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "f9df5c507b684ae39cb6cc2b60d0aed5"
format_stamp: "Formatted at 2026-08-06 19:45:31 on dist-test-slave-77v9"
server_key: "9b7b629aee541915c5ed36f8b4769d2f"
server_key_iv: "40908481aa453113567b9bcb6d76deef"
server_key_version: "encryptionkey@0"
I20260806 19:45:31.621978 3585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "f9df5c507b684ae39cb6cc2b60d0aed5"
format_stamp: "Formatted at 2026-08-06 19:45:31 on dist-test-slave-77v9"
server_key: "9b7b629aee541915c5ed36f8b4769d2f"
server_key_iv: "40908481aa453113567b9bcb6d76deef"
server_key_version: "encryptionkey@0"
I20260806 19:45:31.628434 3585 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260806 19:45:31.632833 3600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:31.634634 3585 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:45:31.634887 3585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "f9df5c507b684ae39cb6cc2b60d0aed5"
format_stamp: "Formatted at 2026-08-06 19:45:31 on dist-test-slave-77v9"
server_key: "9b7b629aee541915c5ed36f8b4769d2f"
server_key_iv: "40908481aa453113567b9bcb6d76deef"
server_key_version: "encryptionkey@0"
I20260806 19:45:31.635136 3585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:31.679768 3585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:31.687160 3585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:31.687837 3585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:31.711184 3585 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:38851
I20260806 19:45:31.711210 3653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:38851 every 8 connection(s)
I20260806 19:45:31.712872 3585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:45:31.717356 3654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:31.719658 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 3585
I20260806 19:45:31.719966 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:45:31.720675 22313 external_mini_cluster.cc:1428] Setting key b15148b0c47e333fefc71cd29e5cb705
I20260806 19:45:31.725535 3654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:45:31 dist-test-slave-77v9 krb5kdc[3572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045531, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.202.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.125@KRBTEST.COM" created.
I20260806 19:45:31.762785 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:31.730067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:38851 (local address 127.0.0.1:52620)
0806 19:45:31.733149 (+ 3082us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:31.733207 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:31.735333 (+ 2126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:31.736426 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:31.736433 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:31.737308 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:31.737698 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:31.737719 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:31.739521 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:31.739532 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:31.741184 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:31.741209 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:31.742422 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:31.742431 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:31.742672 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:31.743735 (+ 1063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:31.743775 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:31.747772 (+ 3997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:31.754155 (+ 6383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:31.754169 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:31.754184 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:31.754732 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:31.755212 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:31.755217 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:31.755221 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:31.755428 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:31.758319 (+ 2891us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:31.758354 (+ 35us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:31.759375 (+ 1021us) client_negotiation.cc:769] Sending connection context
0806 19:45:31.759865 (+ 490us) client_negotiation.cc:240] Negotiation successful
0806 19:45:31.760633 (+ 768us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2836,"mutex_wait_us":142,"thread_start_us":1993,"threads_started":1}
I20260806 19:45:31.762683 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:31.730066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52620 (local address 127.21.202.126:38851)
0806 19:45:31.732684 (+ 2618us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:31.732698 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:31.733368 (+ 670us) server_negotiation.cc:407] Connection header received
0806 19:45:31.735828 (+ 2460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:31.735835 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:31.736130 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:31.736389 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:31.737941 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:31.739318 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:31.741353 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:31.742119 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:31.742360 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:31.748091 (+ 5731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:31.748101 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:31.748115 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:31.748156 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:31.753943 (+ 5787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:31.754902 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:31.754909 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:31.754911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:31.755077 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:31.755554 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:31.755561 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:31.755569 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:31.756402 (+ 833us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:31.758338 (+ 1936us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:31.760083 (+ 1745us) server_negotiation.cc:299] Negotiation successful
0806 19:45:31.760507 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":2377,"spinlock_wait_cycles":57088,"thread_start_us":2050,"threads_started":1}
I20260806 19:45:31.769024 3654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:45:31.771683 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:31.770513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33483 (local address 127.0.0.1:45162)
0806 19:45:31.770961 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:31.771421 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222}
W20260806 19:45:31.772625 3601 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
I20260806 19:45:31.778107 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:31.776722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33483 (local address 127.0.0.1:45174)
0806 19:45:31.777041 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:31.777196 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260806 19:45:31.778762 3654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:33483: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.125.keytab.
Entry for principal kudu/127.21.202.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.125.keytab.
W20260806 19:45:31.822775 22313 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.202.125: real 0.101s user 0.006s sys 0.022s
I20260806 19:45:31.823928 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.125:33483
--webserver_interface=127.21.202.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.125.keytab
--principal=kudu/127.21.202.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:33813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:38851,127.21.202.125:33483,127.21.202.124:41257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:45:31.828428 3654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } attempt: 1
I20260806 19:45:31.830219 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:31.829462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33483 (local address 127.0.0.1:45182)
0806 19:45:31.829786 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:31.830000 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 19:45:31.831727 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:31.831172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33483 (local address 127.0.0.1:45198)
0806 19:45:31.831402 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:31.831518 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 19:45:31.832032 3654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:33483: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
I20260806 19:45:31.900406 3654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } attempt: 2
I20260806 19:45:31.901957 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:31.901328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33483 (local address 127.0.0.1:45210)
0806 19:45:31.901676 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:31.901774 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260806 19:45:31.903259 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:31.902796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33483 (local address 127.0.0.1:45218)
0806 19:45:31.902973 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:31.903105 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 19:45:31.903765 3654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:33483: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
I20260806 19:45:31.995155 3654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } attempt: 3
I20260806 19:45:31.996650 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:31.996010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33483 (local address 127.0.0.1:45230)
0806 19:45:31.996388 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:31.996510 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230}
I20260806 19:45:31.998119 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:31.997572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33483 (local address 127.0.0.1:45246)
0806 19:45:31.997855 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:31.997937 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260806 19:45:31.998541 3654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:33483: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
I20260806 19:45:32.141954 3654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } attempt: 4
I20260806 19:45:32.143687 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.143015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33483 (local address 127.0.0.1:45262)
0806 19:45:32.143421 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.143523 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263}
I20260806 19:45:32.145033 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.144469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33483 (local address 127.0.0.1:45272)
0806 19:45:32.144633 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.144740 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 19:45:32.145289 3654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:33483: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33483: connect: Connection refused (error 111)
W20260806 19:45:32.267634 3661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:32.268083 3661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:32.268401 3661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:32.277161 3661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:45:32.277328 3661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:32.277413 3661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:32.277513 3661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:45:32.277611 3661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:45:32.288581 3661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:33813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:38851,127.21.202.125:33483,127.21.202.124:41257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.125.keytab
--principal=kudu/127.21.202.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.125:33483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.202.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:32.290231 3661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:32.291931 3661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:32.305945 3666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:32.306005 3669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:32.306041 3667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:32.307178 3661 server_base.cc:1034] running on GCE node
I20260806 19:45:32.308319 3661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:32.310099 3661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:32.311417 3661 hybrid_clock.cc:648] HybridClock initialized: now 1786045532311432 us; error 98 us; skew 500 ppm
I20260806 19:45:32.311973 3661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:32 dist-test-slave-77v9 krb5kdc[3572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045532, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:32.316114 3661 init.cc:377] Logged in from keytab as kudu/127.21.202.125@KRBTEST.COM (short username kudu)
I20260806 19:45:32.318126 3661 webserver.cc:460] Webserver started at http://127.21.202.125:45943/ using document root <none> and password file <none>
I20260806 19:45:32.318957 3661 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:32.319128 3661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:32.319600 3661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:45:32.323752 3661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/data/instance:
uuid: "294c95f08ef943929437362cd7d353f9"
format_stamp: "Formatted at 2026-08-06 19:45:32 on dist-test-slave-77v9"
server_key: "7aa8b5656741ccb420788e567ca46e13"
server_key_iv: "a1103b0967b792dfbdf8ed063bdf66d5"
server_key_version: "encryptionkey@0"
I20260806 19:45:32.324746 3661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/wal/instance:
uuid: "294c95f08ef943929437362cd7d353f9"
format_stamp: "Formatted at 2026-08-06 19:45:32 on dist-test-slave-77v9"
server_key: "7aa8b5656741ccb420788e567ca46e13"
server_key_iv: "a1103b0967b792dfbdf8ed063bdf66d5"
server_key_version: "encryptionkey@0"
I20260806 19:45:32.330752 3661 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260806 19:45:32.334919 3676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:32.336632 3661 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 19:45:32.336954 3661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/wal
uuid: "294c95f08ef943929437362cd7d353f9"
format_stamp: "Formatted at 2026-08-06 19:45:32 on dist-test-slave-77v9"
server_key: "7aa8b5656741ccb420788e567ca46e13"
server_key_iv: "a1103b0967b792dfbdf8ed063bdf66d5"
server_key_version: "encryptionkey@0"
I20260806 19:45:32.337180 3661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:32.355702 3661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:32.361217 3661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:32.361558 3661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:32.387550 3661 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.125:33483
I20260806 19:45:32.387598 3728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.125:33483 every 8 connection(s)
I20260806 19:45:32.389242 3661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/data/info.pb
I20260806 19:45:32.393803 3729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:32.397511 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 3661
I20260806 19:45:32.397866 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-1/wal/instance
I20260806 19:45:32.399134 22313 external_mini_cluster.cc:1428] Setting key 50829f4f4d6be69e0a52a47c568e4439
I20260806 19:45:32.403764 3729 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:45:32 dist-test-slave-77v9 krb5kdc[3572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045532, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.125@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.202.124@KRBTEST.COM; defaulting to no policy
I20260806 19:45:32.435276 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.407335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52626 (local address 127.21.202.126:38851)
0806 19:45:32.407939 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:32.407943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:32.409637 (+ 1694us) server_negotiation.cc:407] Connection header received
0806 19:45:32.413100 (+ 3463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:32.413108 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:32.413236 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:32.413377 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:32.415331 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:32.416535 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:32.418532 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:32.419284 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:32.419441 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:32.424693 (+ 5252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:32.424704 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:32.424712 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:32.424780 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:32.429682 (+ 4902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:32.431171 (+ 1489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:32.431179 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:32.431185 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:32.431454 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:32.432270 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:32.432275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:32.432279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:32.432741 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:32.432990 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:32.434566 (+ 1576us) server_negotiation.cc:299] Negotiation successful
0806 19:45:32.434852 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":308,"threads_started":1}
I20260806 19:45:32.436261 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.408137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:38851 (local address 127.0.0.1:52626)
0806 19:45:32.409437 (+ 1300us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.409493 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:32.412652 (+ 3159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:32.413678 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:32.413689 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:32.414347 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:32.415030 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:32.415050 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:32.416761 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:32.416766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:32.418346 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:32.418367 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:32.419550 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:32.419556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:32.419799 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:32.421049 (+ 1250us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:32.421089 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:32.424342 (+ 3253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:32.429986 (+ 5644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:32.430002 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:32.430031 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:32.430901 (+ 870us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:32.431627 (+ 726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:32.431633 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:32.431640 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:32.432088 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:32.433062 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:32.433074 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:32.433676 (+ 602us) client_negotiation.cc:769] Sending connection context
0806 19:45:32.434052 (+ 376us) client_negotiation.cc:240] Negotiation successful
0806 19:45:32.434644 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"thread_start_us":692,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.124@KRBTEST.COM" created.
I20260806 19:45:32.438951 3729 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:45:32.444629 3654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } attempt: 5
Aug 06 19:45:32 dist-test-slave-77v9 krb5kdc[3572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045532, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.125@KRBTEST.COM for kudu/127.21.202.125@KRBTEST.COM
Aug 06 19:45:32 dist-test-slave-77v9 krb5kdc[3572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045531, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for kudu/127.21.202.125@KRBTEST.COM
I20260806 19:45:32.460629 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.440015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33483 (local address 127.0.0.1:45278)
0806 19:45:32.440197 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.440218 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:32.440384 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:32.441442 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:32.441447 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:32.441957 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:32.442096 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:32.442104 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:32.443700 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:32.443708 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:32.444861 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:32.444880 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:32.446767 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:32.446771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:32.446966 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:32.448427 (+ 1461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:32.448450 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:32.451223 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:32.456652 (+ 5429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:32.456658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:32.456663 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:32.457380 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:32.457940 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:32.457946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:32.457951 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:32.458183 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:32.459112 (+ 929us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:32.459121 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:32.459330 (+ 209us) client_negotiation.cc:769] Sending connection context
0806 19:45:32.459500 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 19:45:32.459904 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 19:45:32.460659 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.440138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45278 (local address 127.21.202.125:33483)
0806 19:45:32.440858 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:32.440867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:32.440900 (+ 33us) server_negotiation.cc:407] Connection header received
0806 19:45:32.440961 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:32.440967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:32.441250 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:32.441454 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:32.442267 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:32.443558 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:32.445055 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:32.446244 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:32.446552 (+ 308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:32.451522 (+ 4970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:32.451541 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:32.451563 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:32.451607 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:32.456254 (+ 4647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:32.457592 (+ 1338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:32.457602 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:32.457605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:32.457795 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:32.458337 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:32.458344 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:32.458349 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:32.458916 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:32.459163 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:32.459727 (+ 564us) server_negotiation.cc:299] Negotiation successful
0806 19:45:32.460118 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":465,"thread_start_us":325,"threads_started":1}
I20260806 19:45:32.464800 3729 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:45:32.467105 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.465892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42224)
0806 19:45:32.466380 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.466817 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209}
W20260806 19:45:32.467460 3680 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
I20260806 19:45:32.470350 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.445427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33483 (local address 127.0.0.1:45280)
0806 19:45:32.445754 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.445777 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:32.446004 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:32.447267 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:32.447272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:32.447663 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:32.447849 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:32.447858 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:32.449859 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:32.449869 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:32.451302 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:32.451332 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:32.452426 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:32.452433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:32.452672 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:32.454068 (+ 1396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:32.454104 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:32.457417 (+ 3313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:32.464366 (+ 6949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:32.464375 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:32.464381 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:32.466638 (+ 2257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:32.467186 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:32.467191 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:32.467194 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:32.467400 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:32.468538 (+ 1138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:32.468561 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:32.468922 (+ 361us) client_negotiation.cc:769] Sending connection context
0806 19:45:32.469341 (+ 419us) client_negotiation.cc:240] Negotiation successful
0806 19:45:32.469807 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189}
I20260806 19:45:32.470417 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.469760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42234)
0806 19:45:32.470097 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.470223 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260806 19:45:32.470664 3729 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:41257: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
I20260806 19:45:32.472292 3735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.445672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45280 (local address 127.21.202.125:33483)
0806 19:45:32.446846 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:32.446861 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:32.446892 (+ 31us) server_negotiation.cc:407] Connection header received
0806 19:45:32.446964 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:32.446972 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:32.447060 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:32.447200 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:32.448041 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:32.449703 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:32.451483 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:32.452197 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:32.455377 (+ 3180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:32.457733 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:32.457746 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:32.457756 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:32.457807 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:32.464157 (+ 6350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:32.466804 (+ 2647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:32.466826 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:32.466833 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:32.467052 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:32.467557 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:32.467562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:32.467567 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:32.468155 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:32.471253 (+ 3098us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:32.471452 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 19:45:32.471755 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1021,"thread_start_us":589,"threads_started":1}
I20260806 19:45:32.473747 3654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:45:32.475289 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.474713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42246)
0806 19:45:32.474968 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.475095 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 19:45:32.477139 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.476649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42248)
0806 19:45:32.476870 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.476967 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 19:45:32.477512 3654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:41257: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.124.keytab.
Entry for principal kudu/127.21.202.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.124.keytab.
I20260806 19:45:32.489960 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.124:41257
--webserver_interface=127.21.202.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.124.keytab
--principal=kudu/127.21.202.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:33813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:38851,127.21.202.125:33483,127.21.202.124:41257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:45:32.520207 3729 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } attempt: 1
I20260806 19:45:32.522039 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.521446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42262)
0806 19:45:32.521693 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.521808 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260806 19:45:32.523150 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.522774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42268)
0806 19:45:32.522961 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.523031 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 19:45:32.523357 3729 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:41257: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
I20260806 19:45:32.528865 3654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } attempt: 1
I20260806 19:45:32.530030 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.529575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42282)
0806 19:45:32.529776 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.529876 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 19:45:32.531832 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.530827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42294)
0806 19:45:32.531282 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.531429 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281}
W20260806 19:45:32.532084 3654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:41257: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
I20260806 19:45:32.591660 3729 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } attempt: 2
I20260806 19:45:32.594122 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.592613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42304)
0806 19:45:32.593203 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.593412 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260806 19:45:32.596045 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.595447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42320)
0806 19:45:32.595672 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.595809 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 19:45:32.596364 3729 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:41257: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
I20260806 19:45:32.600343 3654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } attempt: 2
I20260806 19:45:32.601780 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.601104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42328)
0806 19:45:32.601404 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.601518 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 19:45:32.603216 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.602837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42334)
0806 19:45:32.603007 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.603102 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 19:45:32.603500 3654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:41257: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
I20260806 19:45:32.687660 3729 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } attempt: 3
I20260806 19:45:32.689150 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.688475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42336)
0806 19:45:32.688751 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.688926 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 19:45:32.690943 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.690192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42348)
0806 19:45:32.690586 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.690718 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218}
W20260806 19:45:32.691330 3729 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:41257: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
I20260806 19:45:32.709843 3654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } attempt: 3
I20260806 19:45:32.711488 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.710785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42352)
0806 19:45:32.711066 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.711190 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260806 19:45:32.713454 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.712762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42362)
0806 19:45:32.713187 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.713299 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311}
W20260806 19:45:32.713743 3654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:41257: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
I20260806 19:45:32.834609 3729 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } attempt: 4
I20260806 19:45:32.835989 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.835476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42378)
0806 19:45:32.835698 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.835792 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 19:45:32.837311 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.836942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42380)
0806 19:45:32.837111 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.837183 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 19:45:32.837527 3729 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:41257: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
I20260806 19:45:32.891111 3654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } attempt: 4
I20260806 19:45:32.892449 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.891871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42384)
0806 19:45:32.892116 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.892225 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 19:45:32.893802 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:32.893396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42396)
0806 19:45:32.893563 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:32.893648 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 19:45:32.894070 3654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:41257: Network error: Client connection negotiation failed: client connection to 127.21.202.124:41257: connect: Connection refused (error 111)
W20260806 19:45:32.900709 3738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:32.901185 3738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:32.901352 3738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:32.909396 3738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:45:32.909572 3738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:32.909678 3738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:32.909788 3738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:45:32.909879 3738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:45:32.920615 3738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:33813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:38851,127.21.202.125:33483,127.21.202.124:41257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.124.keytab
--principal=kudu/127.21.202.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.124:41257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.202.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:32.922152 3738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:32.923571 3738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:32.936177 3743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:32.936278 3747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:32.936411 3744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:32.937634 3738 server_base.cc:1034] running on GCE node
I20260806 19:45:32.938591 3738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:32.940271 3738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:32.941676 3738 hybrid_clock.cc:648] HybridClock initialized: now 1786045532941616 us; error 79 us; skew 500 ppm
I20260806 19:45:32.942174 3738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:32 dist-test-slave-77v9 krb5kdc[3572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045532, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:32.946398 3738 init.cc:377] Logged in from keytab as kudu/127.21.202.124@KRBTEST.COM (short username kudu)
I20260806 19:45:32.948318 3738 webserver.cc:460] Webserver started at http://127.21.202.124:37211/ using document root <none> and password file <none>
I20260806 19:45:32.949234 3738 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:32.949406 3738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:32.949784 3738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:45:32.953514 3738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/data/instance:
uuid: "db317b1052c8470cbf96fc4e42640334"
format_stamp: "Formatted at 2026-08-06 19:45:32 on dist-test-slave-77v9"
server_key: "45d4fecfc19a44293dc556a7b6d663b1"
server_key_iv: "54664ffc025e088b3cd6b36f006a13ae"
server_key_version: "encryptionkey@0"
I20260806 19:45:32.954478 3738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/wal/instance:
uuid: "db317b1052c8470cbf96fc4e42640334"
format_stamp: "Formatted at 2026-08-06 19:45:32 on dist-test-slave-77v9"
server_key: "45d4fecfc19a44293dc556a7b6d663b1"
server_key_iv: "54664ffc025e088b3cd6b36f006a13ae"
server_key_version: "encryptionkey@0"
I20260806 19:45:32.961114 3738 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 19:45:32.965389 3754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:32.967517 3738 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:45:32.967959 3738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/wal
uuid: "db317b1052c8470cbf96fc4e42640334"
format_stamp: "Formatted at 2026-08-06 19:45:32 on dist-test-slave-77v9"
server_key: "45d4fecfc19a44293dc556a7b6d663b1"
server_key_iv: "54664ffc025e088b3cd6b36f006a13ae"
server_key_version: "encryptionkey@0"
I20260806 19:45:32.968295 3738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:32.992002 3738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:32.997275 3738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:32.997606 3738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:33.023816 3738 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.124:41257
I20260806 19:45:33.023969 3808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.124:41257 every 8 connection(s)
I20260806 19:45:33.025529 3738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/data/info.pb
I20260806 19:45:33.030689 3809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:33.034668 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 3738
I20260806 19:45:33.034965 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786045310306758-22313-0/minicluster-data/master-2/wal/instance
I20260806 19:45:33.035713 22313 external_mini_cluster.cc:1428] Setting key 6ffed4e5ebb06e0317ef7c8d9cfc499b
I20260806 19:45:33.039892 3809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:45:33 dist-test-slave-77v9 krb5kdc[3572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.124@KRBTEST.COM
Aug 06 19:45:33 dist-test-slave-77v9 krb5kdc[3572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045532, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.124@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:33.065642 3814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.044557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52630 (local address 127.21.202.126:38851)
0806 19:45:33.045371 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:33.045377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:33.045959 (+ 582us) server_negotiation.cc:407] Connection header received
0806 19:45:33.046178 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:33.046184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:33.046266 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:33.046387 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:33.047669 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.048441 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.050288 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.050754 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.050913 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.054972 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:33.054978 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:33.054981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:33.055016 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:33.061208 (+ 6192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.062296 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.062302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.062304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.062531 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.063191 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.063196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.063198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.063763 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:33.064010 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:33.064747 (+ 737us) server_negotiation.cc:299] Negotiation successful
0806 19:45:33.065100 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":466,"threads_started":1}
I20260806 19:45:33.066692 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.038085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42410 (local address 127.21.202.124:41257)
0806 19:45:33.039057 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:33.039076 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:33.039132 (+ 56us) server_negotiation.cc:407] Connection header received
0806 19:45:33.041453 (+ 2321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:33.041495 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:33.042152 (+ 657us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:33.042673 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:33.044533 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.046750 (+ 2217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.048191 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.048969 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.049194 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.054085 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:33.054095 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:33.054113 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:33.054155 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:33.061512 (+ 7357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.062564 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.062584 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.062591 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.062797 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.063446 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.063451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.063454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.064281 (+ 827us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:33.064536 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:33.065075 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 19:45:33.065535 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":684,"thread_start_us":421,"threads_started":1}
I20260806 19:45:33.066740 3815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.044555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:38851 (local address 127.0.0.1:52630)
0806 19:45:33.045767 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:33.045832 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:33.046025 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:33.046477 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:33.046482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:33.047219 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:33.047496 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:33.047506 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.048627 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.048633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:33.050080 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:33.050106 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.051034 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.051043 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:33.051224 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.051905 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:33.051937 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:33.054749 (+ 2812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:33.061448 (+ 6699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:33.061462 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:33.061474 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:33.062109 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:33.062716 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:33.062720 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:33.062722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:33.063023 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:33.063981 (+ 958us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:33.063988 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:33.064265 (+ 277us) client_negotiation.cc:769] Sending connection context
0806 19:45:33.064633 (+ 368us) client_negotiation.cc:240] Negotiation successful
0806 19:45:33.065320 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"thread_start_us":729,"threads_started":1}
I20260806 19:45:33.069128 3809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:45:33 dist-test-slave-77v9 krb5kdc[3572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045532, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.124@KRBTEST.COM for kudu/127.21.202.125@KRBTEST.COM
Aug 06 19:45:33 dist-test-slave-77v9 krb5kdc[3572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:33.086730 3815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.069979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33483 (local address 127.0.0.1:45288)
0806 19:45:33.070249 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:33.070273 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:33.070410 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:33.071151 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:33.071155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:33.071512 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:33.071663 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:33.071671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.073473 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.073482 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:33.074959 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:33.074977 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.075787 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.075792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:33.076023 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.076477 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:33.076493 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:33.078912 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:33.083490 (+ 4578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:33.083495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:33.083497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:33.083998 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:33.084500 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:33.084505 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:33.084508 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:33.084707 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:33.085700 (+ 993us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:33.085705 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:33.085965 (+ 260us) client_negotiation.cc:769] Sending connection context
0806 19:45:33.086141 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 19:45:33.086407 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260806 19:45:33.087061 3816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.070085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45288 (local address 127.21.202.125:33483)
0806 19:45:33.070859 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:33.070863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:33.070879 (+ 16us) server_negotiation.cc:407] Connection header received
0806 19:45:33.070926 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:33.070930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:33.071026 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:33.071139 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:33.071828 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.073163 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.075153 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.075611 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.075779 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.079159 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:33.079165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:33.079168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:33.079211 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:33.083264 (+ 4053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.084151 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.084157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.084161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.084353 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.085001 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.085005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.085011 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.085410 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:33.085618 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:33.086323 (+ 705us) server_negotiation.cc:299] Negotiation successful
0806 19:45:33.086629 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":520,"threads_started":1}
I20260806 19:45:33.087791 3814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.070439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52640 (local address 127.21.202.126:38851)
0806 19:45:33.070637 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:33.070644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:33.070665 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:45:33.070813 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:33.070816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:33.070885 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:33.070967 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:33.072565 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.073471 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.074810 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.075223 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.075367 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.080242 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:33.080251 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:33.080257 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:33.080311 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:33.084214 (+ 3903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.085248 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.085253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.085257 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.085416 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.086046 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.086051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.086053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.086531 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:33.086864 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:33.087258 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 19:45:33.087487 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 19:45:33.088464 3809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:45:33 dist-test-slave-77v9 krb5kdc[3572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045532, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.124@KRBTEST.COM for kudu/127.21.202.124@KRBTEST.COM
Aug 06 19:45:33 dist-test-slave-77v9 krb5kdc[3572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.125@KRBTEST.COM
I20260806 19:45:33.105823 3815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.089749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42422)
0806 19:45:33.089962 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:33.089986 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:33.090131 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:33.090662 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:33.090667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:33.091124 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:33.091292 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:33.091301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.092757 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.092765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:33.093998 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:33.094013 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.094888 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.094958 (+ 70us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:33.095155 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.095909 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:33.095944 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:33.098805 (+ 2861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:33.102289 (+ 3484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:33.102296 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:33.102299 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:33.103133 (+ 834us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:33.103819 (+ 686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:33.103824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:33.103827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:33.104023 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:33.104946 (+ 923us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:33.104950 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:33.105088 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 19:45:33.105242 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 19:45:33.105500 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 19:45:33.106424 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.089504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42422 (local address 127.21.202.124:41257)
0806 19:45:33.089830 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:33.089835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:33.090071 (+ 236us) server_negotiation.cc:407] Connection header received
0806 19:45:33.090382 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:33.090391 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:33.090487 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:33.090600 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:33.091757 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.092570 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.094171 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.094704 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.094800 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.099083 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:33.099088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:33.099091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:33.099123 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:33.101977 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.103336 (+ 1359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.103343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.103349 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.103597 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.104298 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.104307 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.104311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.104729 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:33.104945 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:33.105380 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 19:45:33.105709 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":93}
I20260806 19:45:33.109905 3816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.090801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45296 (local address 127.21.202.125:33483)
0806 19:45:33.091042 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:33.091048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:33.091067 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:45:33.091234 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:33.091240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:33.091320 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:33.091765 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:33.093407 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.094201 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.095988 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.096452 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.096563 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.101964 (+ 5401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:33.101971 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:33.101975 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:33.102023 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:33.106273 (+ 4250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.107300 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.107304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.107307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.107456 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.108135 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.108140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.108143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.108563 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:33.108982 (+ 419us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:33.109290 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 19:45:33.109551 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 19:45:33.112746 3809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334: Bootstrap starting.
I20260806 19:45:33.115937 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:45:33.117004 3809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:33.118402 3809 log.cc:824] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:33.121945 3809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334: No bootstrap required, opened a new log
I20260806 19:45:33.136729 3729 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } attempt: 5
I20260806 19:45:33.137688 3809 raft_consensus.cc:348] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } }
I20260806 19:45:33.138284 3809 raft_consensus.cc:374] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:33.138442 3809 raft_consensus.cc:729] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db317b1052c8470cbf96fc4e42640334, State: Initialized, Role: FOLLOWER
I20260806 19:45:33.139309 3809 consensus_queue.cc:260] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } }
I20260806 19:45:33.154170 3826 sys_catalog.cc:455] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } } }
I20260806 19:45:33.154826 3826 sys_catalog.cc:458] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:45:33.156100 3809 sys_catalog.cc:564] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:45:33.158936 3814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.122145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52650 (local address 127.21.202.126:38851)
0806 19:45:33.122557 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:33.122566 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:33.127546 (+ 4980us) server_negotiation.cc:407] Connection header received
0806 19:45:33.127745 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:33.127752 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:33.127985 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:33.128118 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:33.130725 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.131914 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.134772 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.135447 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.135547 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.141875 (+ 6328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:33.141889 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:33.141899 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:33.141977 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:33.151087 (+ 9110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.152984 (+ 1897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.152996 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.153003 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.153251 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.154231 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.154237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.154240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.154818 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:33.155174 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:33.157819 (+ 2645us) server_negotiation.cc:299] Negotiation successful
0806 19:45:33.158190 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266}
I20260806 19:45:33.163211 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.122334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.21.202.124:41257)
0806 19:45:33.122760 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:33.122769 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:33.126491 (+ 3722us) server_negotiation.cc:407] Connection header received
0806 19:45:33.126641 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:33.126653 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:33.126769 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:33.126892 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:33.129961 (+ 3069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.131299 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.134290 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.135126 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.135239 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.141442 (+ 6203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:33.141453 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:33.141461 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:33.141524 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:33.147089 (+ 5565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.155318 (+ 8229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.155324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.155327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.155895 (+ 568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.156927 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.156935 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.157281 (+ 346us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.158210 (+ 929us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:33.158434 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:33.162462 (+ 4028us) server_negotiation.cc:299] Negotiation successful
0806 19:45:33.162756 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":266}
I20260806 19:45:33.164558 3816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.122757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45302 (local address 127.21.202.125:33483)
0806 19:45:33.125115 (+ 2358us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:33.125121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:33.125147 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:45:33.125431 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:33.125442 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:33.125555 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:33.125671 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:33.130154 (+ 4483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.131448 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.134594 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.135449 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.135605 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.154578 (+ 18973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:33.154589 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:33.154595 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:33.154656 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:33.160368 (+ 5712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.161454 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.161461 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.161465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.161631 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.162229 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.162237 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.162240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.162723 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:33.163259 (+ 536us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:33.163596 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 19:45:33.163962 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2148}
Aug 06 19:45:33 dist-test-slave-77v9 krb5kdc[3572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045532, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.125@KRBTEST.COM for kudu/127.21.202.124@KRBTEST.COM
I20260806 19:45:33.171396 3654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } attempt: 5
I20260806 19:45:33.181053 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.137421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42438)
0806 19:45:33.137838 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:33.137859 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:33.137975 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:33.156754 (+ 18779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:33.156764 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:33.157673 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:33.157943 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:33.157959 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.160319 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.160329 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:33.162119 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:33.162139 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.166440 (+ 4301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.166450 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:33.166811 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.167621 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:33.167647 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:33.170680 (+ 3033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:33.176641 (+ 5961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:33.176651 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:33.176658 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:33.177309 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:33.177953 (+ 644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:33.177961 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:33.177967 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:33.178225 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:33.179225 (+ 1000us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:33.179234 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:33.179463 (+ 229us) client_negotiation.cc:769] Sending connection context
0806 19:45:33.179795 (+ 332us) client_negotiation.cc:240] Negotiation successful
0806 19:45:33.180299 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277}
I20260806 19:45:33.182643 3825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.138199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42438 (local address 127.21.202.124:41257)
0806 19:45:33.155642 (+ 17443us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:33.155650 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:33.155678 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:45:33.155733 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:33.155739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:33.155835 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:33.157660 (+ 1825us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:33.158521 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.159981 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.165386 (+ 5405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.166208 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.166398 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.170972 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:33.170981 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:33.170988 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:33.171043 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:33.176441 (+ 5398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.177531 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.177540 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.177546 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.177777 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.178405 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.178413 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.178419 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.178945 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:33.179128 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:33.181463 (+ 2335us) server_negotiation.cc:299] Negotiation successful
0806 19:45:33.181885 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":473,"server-negotiator.queue_time_us":17242,"thread_start_us":17766,"threads_started":1}
Aug 06 19:45:33 dist-test-slave-77v9 krb5kdc[3572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045531, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for kudu/127.21.202.124@KRBTEST.COM
W20260806 19:45:33.190431 3837 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:45:33.190653 3837 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:45:33.191056 3729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9: Bootstrap starting.
I20260806 19:45:33.196542 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.172125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:41257 (local address 127.0.0.1:42450)
0806 19:45:33.172777 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:33.172830 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:33.172964 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:33.174692 (+ 1728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:33.174712 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:33.175311 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:33.175593 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:33.175613 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.178457 (+ 2844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.178466 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:33.180314 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:33.180337 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.183156 (+ 2819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.183164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:33.183431 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.184281 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:33.184314 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:33.188162 (+ 3848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:33.192894 (+ 4732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:33.192900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:33.192905 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:33.193617 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:33.194143 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:33.194150 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:33.194155 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:33.194345 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:33.195207 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:33.195214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:33.195562 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 19:45:33.195727 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 19:45:33.196072 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478}
I20260806 19:45:33.197618 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:33.173764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.21.202.124:41257)
0806 19:45:33.173983 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:33.173988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:33.174007 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:45:33.174105 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:33.174112 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:33.174274 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:33.174424 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:33.176950 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.178235 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.180584 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:33.181536 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:33.181741 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:33.188435 (+ 6694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:33.188448 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:33.188455 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:33.188506 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:33.192658 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.193810 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.193817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.193822 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.193984 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:33.194503 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:33.194508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:33.194517 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:33.194980 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:33.195129 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:33.196973 (+ 1844us) server_negotiation.cc:299] Negotiation successful
0806 19:45:33.197247 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 19:45:33.198668 3729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:33.200730 3729 log.cc:824] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:33.178989 3738 master_runner.cc:186] Error getting master registration for 127.21.202.126:38851: OK, instance_id { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" instance_seqno: 1786045531704782 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 19:45:33.202231 3738 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 19:45:33.204519 3654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5: Bootstrap starting.
I20260806 19:45:33.208093 3729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9: No bootstrap required, opened a new log
I20260806 19:45:33.209400 3654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:33.210698 3654 log.cc:824] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:33.214591 3654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5: No bootstrap required, opened a new log
I20260806 19:45:33.219826 3729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } }
I20260806 19:45:33.220327 3729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:33.220456 3729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 294c95f08ef943929437362cd7d353f9, State: Initialized, Role: FOLLOWER
I20260806 19:45:33.221194 3729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } }
I20260806 19:45:33.222288 3841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } } }
I20260806 19:45:33.222841 3841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:45:33.223774 3729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:45:33.226773 3654 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } }
I20260806 19:45:33.227236 3654 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:33.227387 3654 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9df5c507b684ae39cb6cc2b60d0aed5, State: Initialized, Role: FOLLOWER
I20260806 19:45:33.228133 3654 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } }
I20260806 19:45:33.230265 3843 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } } }
I20260806 19:45:33.230789 3843 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:45:33.231691 3654 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 19:45:33.251142 3761 tablet.cc:2369] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:45:33.249337 3585 master_runner.cc:186] Error getting master registration for 127.21.202.125:33483: OK, instance_id { permanent_uuid: "294c95f08ef943929437362cd7d353f9" instance_seqno: 1786045532379622 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 19:45:33.266625 3585 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 19:45:33.269830 3861 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:45:33.270066 3861 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:45:33.283962 3661 master_runner.cc:263] No leader master found from master 294c95f08ef943929437362cd7d353f9
I20260806 19:45:33.284170 3661 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 19:45:33.288302 3866 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:45:33.288679 3866 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:45:33.344475 3841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:45:33.344924 3841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } }
I20260806 19:45:33.347136 3841 leader_election.cc:288] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9df5c507b684ae39cb6cc2b60d0aed5 (127.21.202.126:38851), db317b1052c8470cbf96fc4e42640334 (127.21.202.124:41257)
I20260806 19:45:33.349236 3826 raft_consensus.cc:482] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:45:33.348632 3628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "294c95f08ef943929437362cd7d353f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" is_pre_election: true
I20260806 19:45:33.349541 3826 raft_consensus.cc:504] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } }
I20260806 19:45:33.349771 3628 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294c95f08ef943929437362cd7d353f9 in term 0.
I20260806 19:45:33.350714 3679 leader_election.cc:302] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 294c95f08ef943929437362cd7d353f9, f9df5c507b684ae39cb6cc2b60d0aed5; no voters:
I20260806 19:45:33.351259 3826 leader_election.cc:288] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9df5c507b684ae39cb6cc2b60d0aed5 (127.21.202.126:38851), 294c95f08ef943929437362cd7d353f9 (127.21.202.125:33483)
I20260806 19:45:33.353492 3784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "294c95f08ef943929437362cd7d353f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db317b1052c8470cbf96fc4e42640334" is_pre_election: true
I20260806 19:45:33.354164 3784 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 294c95f08ef943929437362cd7d353f9 in term 0.
I20260806 19:45:33.354686 3704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db317b1052c8470cbf96fc4e42640334" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294c95f08ef943929437362cd7d353f9" is_pre_election: true
I20260806 19:45:33.355275 3704 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db317b1052c8470cbf96fc4e42640334 in term 0.
I20260806 19:45:33.356118 3628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db317b1052c8470cbf96fc4e42640334" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" is_pre_election: true
I20260806 19:45:33.356418 3628 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db317b1052c8470cbf96fc4e42640334 in term 0.
I20260806 19:45:33.357281 3758 leader_election.cc:302] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db317b1052c8470cbf96fc4e42640334, f9df5c507b684ae39cb6cc2b60d0aed5; no voters:
I20260806 19:45:33.357820 3826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:45:33.357965 3826 raft_consensus.cc:482] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:45:33.358070 3826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:33.362082 3841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:45:33.362280 3841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:45:33.362391 3841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [term 0 FOLLOWER]: Advancing to term 1
W20260806 19:45:33.364840 3682 tablet.cc:2369] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:45:33.369752 3826 raft_consensus.cc:504] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } }
I20260806 19:45:33.371255 3628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db317b1052c8470cbf96fc4e42640334" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5"
I20260806 19:45:33.371522 3628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:33.379174 3628 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db317b1052c8470cbf96fc4e42640334 in term 1.
I20260806 19:45:33.380084 3826 leader_election.cc:288] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [CANDIDATE]: Term 1 election: Requested vote from peers f9df5c507b684ae39cb6cc2b60d0aed5 (127.21.202.126:38851), 294c95f08ef943929437362cd7d353f9 (127.21.202.125:33483)
I20260806 19:45:33.380976 3704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db317b1052c8470cbf96fc4e42640334" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294c95f08ef943929437362cd7d353f9"
I20260806 19:45:33.383071 3841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } }
I20260806 19:45:33.383836 3704 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate db317b1052c8470cbf96fc4e42640334 in current term 1: Already voted for candidate 294c95f08ef943929437362cd7d353f9 in this term.
I20260806 19:45:33.388679 3627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "294c95f08ef943929437362cd7d353f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5"
I20260806 19:45:33.390990 3758 leader_election.cc:302] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db317b1052c8470cbf96fc4e42640334, f9df5c507b684ae39cb6cc2b60d0aed5; no voters: 294c95f08ef943929437362cd7d353f9
I20260806 19:45:33.391821 3826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:45:33.393131 3784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "294c95f08ef943929437362cd7d353f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db317b1052c8470cbf96fc4e42640334"
I20260806 19:45:33.393716 3841 leader_election.cc:288] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [CANDIDATE]: Term 1 election: Requested vote from peers f9df5c507b684ae39cb6cc2b60d0aed5 (127.21.202.126:38851), db317b1052c8470cbf96fc4e42640334 (127.21.202.124:41257)
I20260806 19:45:33.399309 3826 raft_consensus.cc:686] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [term 1 LEADER]: Becoming Leader. State: Replica: db317b1052c8470cbf96fc4e42640334, State: Running, Role: LEADER
I20260806 19:45:33.400123 3826 consensus_queue.cc:237] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } }
I20260806 19:45:33.414592 3680 leader_election.cc:302] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 294c95f08ef943929437362cd7d353f9; no voters: db317b1052c8470cbf96fc4e42640334, f9df5c507b684ae39cb6cc2b60d0aed5
I20260806 19:45:33.415620 3841 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 19:45:33.416311 3868 sys_catalog.cc:455] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db317b1052c8470cbf96fc4e42640334. Latest consensus state: current_term: 1 leader_uuid: "db317b1052c8470cbf96fc4e42640334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } } }
I20260806 19:45:33.416867 3868 sys_catalog.cc:458] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:33.423146 3870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:45:33.426757 3870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 19:45:33.442018 3606 tablet.cc:2369] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:45:33.453958 3704 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [term 1 FOLLOWER]: Refusing update from remote peer db317b1052c8470cbf96fc4e42640334: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:45:33.455279 3868 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [LEADER]: Connected to new peer: Peer: permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:45:33.468124 3628 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [term 1 FOLLOWER]: Refusing update from remote peer db317b1052c8470cbf96fc4e42640334: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:45:33.469746 3868 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:45:33.505594 3843 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db317b1052c8470cbf96fc4e42640334. Latest consensus state: current_term: 1 leader_uuid: "db317b1052c8470cbf96fc4e42640334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } } }
I20260806 19:45:33.506006 3843 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:45:33.513579 3871 mvcc.cc:205] Tried to move back new op lower bound from 7315642505011290112 to 7315642504813715456. Current Snapshot: MvccSnapshot[applied={T|T < 7315642505011290112}]
I20260806 19:45:33.517514 3826 sys_catalog.cc:455] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db317b1052c8470cbf96fc4e42640334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } } }
I20260806 19:45:33.518344 3826 sys_catalog.cc:458] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:33.520277 3870 catalog_manager.cc:1347] Generated new cluster ID: 45573379b6554c0286f0960758c3f550
I20260806 19:45:33.520385 3870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:45:33.529886 3873 mvcc.cc:205] Tried to move back new op lower bound from 7315642505011290112 to 7315642504813715456. Current Snapshot: MvccSnapshot[applied={T|T < 7315642505011290112}]
I20260806 19:45:33.542143 3843 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db317b1052c8470cbf96fc4e42640334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } } }
I20260806 19:45:33.542559 3843 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:45:33.554759 3841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db317b1052c8470cbf96fc4e42640334. Latest consensus state: current_term: 1 leader_uuid: "db317b1052c8470cbf96fc4e42640334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } } }
I20260806 19:45:33.555208 3841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:45:33.558480 3826 sys_catalog.cc:455] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db317b1052c8470cbf96fc4e42640334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } } }
I20260806 19:45:33.559233 3826 sys_catalog.cc:458] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:33.566203 3872 mvcc.cc:205] Tried to move back new op lower bound from 7315642505011290112 to 7315642504813715456. Current Snapshot: MvccSnapshot[applied={T|T < 7315642505011290112}]
I20260806 19:45:33.569267 3841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db317b1052c8470cbf96fc4e42640334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9df5c507b684ae39cb6cc2b60d0aed5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 38851 } } peers { permanent_uuid: "294c95f08ef943929437362cd7d353f9" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33483 } } peers { permanent_uuid: "db317b1052c8470cbf96fc4e42640334" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 41257 } } }
I20260806 19:45:33.570757 3841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:45:33.608944 3870 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:45:33.619367 3870 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:45:33.664076 3870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334: Generated new TSK 0
I20260806 19:45:33.666162 3870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:45:34.272210 3861 catalog_manager.cc:1259] Loaded cluster ID: 45573379b6554c0286f0960758c3f550
I20260806 19:45:34.272388 3861 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5: loading cluster ID for follower catalog manager: success
I20260806 19:45:34.275365 3861 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5: acquiring CA information for follower catalog manager: success
I20260806 19:45:34.278074 3861 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f9df5c507b684ae39cb6cc2b60d0aed5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:45:34.291077 3866 catalog_manager.cc:1259] Loaded cluster ID: 45573379b6554c0286f0960758c3f550
I20260806 19:45:34.291280 3866 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9: loading cluster ID for follower catalog manager: success
I20260806 19:45:34.294075 3866 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9: acquiring CA information for follower catalog manager: success
I20260806 19:45:34.296175 3866 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 294c95f08ef943929437362cd7d353f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:45:34.345329 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 3585
I20260806 19:45:34.347041 3649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:34.474268 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 3585
I20260806 19:45:34.500196 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 3661
I20260806 19:45:34.501720 3724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:34.547147 3885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:34.546214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:38851 (local address 127.0.0.1:52666)
0806 19:45:34.546825 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:34.546967 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":423,"threads_started":1}
W20260806 19:45:34.547526 3758 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.202.126:38851: connect: Connection refused (error 111)
I20260806 19:45:34.549157 3885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:34.548701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:38851 (local address 127.0.0.1:52680)
0806 19:45:34.548958 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:34.549034 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260806 19:45:34.549397 3758 consensus_peers.cc:487] T 00000000000000000000000000000000 P db317b1052c8470cbf96fc4e42640334 -> Peer f9df5c507b684ae39cb6cc2b60d0aed5 (127.21.202.126:38851): Couldn't send request to peer f9df5c507b684ae39cb6cc2b60d0aed5. Status: Network error: Client connection negotiation failed: client connection to 127.21.202.126:38851: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 19:45:34.622849 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 3661
I20260806 19:45:34.650023 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 3738
I20260806 19:45:34.651682 3804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:34.772754 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 3738
2026-08-06T19:45:34Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5295 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:45:34 dist-test-slave-77v9 krb5kdc[3890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:45:34 dist-test-slave-77v9 krb5kdc[3890](info): set up 2 sockets
Aug 06 19:45:34 dist-test-slave-77v9 krb5kdc[3890](info): commencing operation
krb5kdc: starting...
W20260806 19:45:36.044091 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:45:36 dist-test-slave-77v9 krb5kdc[3890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:45:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:45:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:45:36.402591 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:34403
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:45763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:34403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:45:36.839033 3904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:36.839380 3904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:36.839552 3904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:36.847268 3904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:45:36.847409 3904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:36.847457 3904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:36.847591 3904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:45:36.847710 3904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:45:36.858412 3904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:34403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:34403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:36.859928 3904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:36.861424 3904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:36.877343 3910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:36.877259 3913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:36.877624 3904 server_base.cc:1034] running on GCE node
W20260806 19:45:36.877370 3911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:36.878903 3904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:36.880601 3904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:36.881875 3904 hybrid_clock.cc:648] HybridClock initialized: now 1786045536881807 us; error 66 us; skew 500 ppm
I20260806 19:45:36.882319 3904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:36 dist-test-slave-77v9 krb5kdc[3890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045536, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:36.886334 3904 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:45:36.888547 3904 webserver.cc:460] Webserver started at http://127.21.202.126:34463/ using document root <none> and password file <none>
I20260806 19:45:36.889561 3904 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:36.889761 3904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:36.890192 3904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:45:36.894393 3904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "ca8a467223f44f76b5adc9fa0112f2ee"
format_stamp: "Formatted at 2026-08-06 19:45:36 on dist-test-slave-77v9"
server_key: "bc403898fd230ba87c8ba1841d9d2bd3"
server_key_iv: "4a347dfc08cae8aedfbc2cef901c61db"
server_key_version: "encryptionkey@0"
I20260806 19:45:36.895511 3904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "ca8a467223f44f76b5adc9fa0112f2ee"
format_stamp: "Formatted at 2026-08-06 19:45:36 on dist-test-slave-77v9"
server_key: "bc403898fd230ba87c8ba1841d9d2bd3"
server_key_iv: "4a347dfc08cae8aedfbc2cef901c61db"
server_key_version: "encryptionkey@0"
I20260806 19:45:36.902345 3904 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 19:45:36.907132 3920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:36.909085 3904 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:45:36.909323 3904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "ca8a467223f44f76b5adc9fa0112f2ee"
format_stamp: "Formatted at 2026-08-06 19:45:36 on dist-test-slave-77v9"
server_key: "bc403898fd230ba87c8ba1841d9d2bd3"
server_key_iv: "4a347dfc08cae8aedfbc2cef901c61db"
server_key_version: "encryptionkey@0"
I20260806 19:45:36.909585 3904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:36.955932 3904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:36.961097 3904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:36.961580 3904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:36.984447 3904 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:34403
I20260806 19:45:36.984635 3972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:34403 every 8 connection(s)
I20260806 19:45:36.986227 3904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:45:36.992596 3973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:36.994333 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 3904
I20260806 19:45:36.994640 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:45:36.995352 22313 external_mini_cluster.cc:1428] Setting key 966a12b2d709218256a18bae37b701f9
I20260806 19:45:37.011662 3973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee: Bootstrap starting.
Aug 06 19:45:37 dist-test-slave-77v9 krb5kdc[3890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:37.017536 3973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:37.019192 3973 log.cc:824] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:37.023005 3973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee: No bootstrap required, opened a new log
I20260806 19:45:37.029659 3976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:36.997696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48976 (local address 127.21.202.126:34403)
0806 19:45:36.998710 (+ 1014us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:36.998723 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:36.998772 (+ 49us) server_negotiation.cc:407] Connection header received
0806 19:45:37.000493 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:37.000519 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:37.001123 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:37.001573 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:37.004313 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:37.006024 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:37.009328 (+ 3304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:37.011350 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:37.011630 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:37.019271 (+ 7641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:37.019282 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:37.019301 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:37.019395 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:37.023746 (+ 4351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:37.025011 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:37.025019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:37.025028 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:37.025241 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:37.025832 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:37.025896 (+ 64us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:37.025899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:37.026666 (+ 767us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:37.026943 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:37.027623 (+ 680us) server_negotiation.cc:299] Negotiation successful
0806 19:45:37.028186 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":465,"threads_started":1}
I20260806 19:45:37.033402 3973 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca8a467223f44f76b5adc9fa0112f2ee" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34403 } }
I20260806 19:45:37.033836 3973 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:37.033959 3973 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca8a467223f44f76b5adc9fa0112f2ee, State: Initialized, Role: FOLLOWER
I20260806 19:45:37.034578 3973 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca8a467223f44f76b5adc9fa0112f2ee" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34403 } }
I20260806 19:45:37.034816 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:45:37.034929 3973 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:45:37.035081 3973 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:45:37.035260 3973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:37.037994 3973 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca8a467223f44f76b5adc9fa0112f2ee" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34403 } }
I20260806 19:45:37.039083 3973 leader_election.cc:302] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca8a467223f44f76b5adc9fa0112f2ee; no voters:
I20260806 19:45:37.039856 3973 leader_election.cc:288] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:45:37.040103 3979 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:45:37.040884 3979 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [term 1 LEADER]: Becoming Leader. State: Replica: ca8a467223f44f76b5adc9fa0112f2ee, State: Running, Role: LEADER
I20260806 19:45:37.041442 3979 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca8a467223f44f76b5adc9fa0112f2ee" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34403 } }
I20260806 19:45:37.042256 3973 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:45:37.045022 3980 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca8a467223f44f76b5adc9fa0112f2ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca8a467223f44f76b5adc9fa0112f2ee" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34403 } } }
I20260806 19:45:37.045820 3980 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:37.045233 3981 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca8a467223f44f76b5adc9fa0112f2ee. Latest consensus state: current_term: 1 leader_uuid: "ca8a467223f44f76b5adc9fa0112f2ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca8a467223f44f76b5adc9fa0112f2ee" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34403 } } }
I20260806 19:45:37.047282 3981 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:37.048252 3986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:45:37.056051 3986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:45:37.067808 3986 catalog_manager.cc:1347] Generated new cluster ID: 5db8fc0b987e40d6ae7378c42f1a6838
I20260806 19:45:37.068013 3986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:45:37.128103 3986 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:45:37.129968 3986 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:45:37.140544 3986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca8a467223f44f76b5adc9fa0112f2ee: Generated new TSK 0
I20260806 19:45:37.141793 3986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:45:37.200511 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 3904
I20260806 19:45:37.202047 3968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:37.311892 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 3904
2026-08-06T19:45:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2534 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:45:37 dist-test-slave-77v9 krb5kdc[4003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:45:37 dist-test-slave-77v9 krb5kdc[4003](info): set up 2 sockets
Aug 06 19:45:37 dist-test-slave-77v9 krb5kdc[4003](info): commencing operation
krb5kdc: starting...
W20260806 19:45:38.566742 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:45:38 dist-test-slave-77v9 krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:45:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:45:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:45:38.931166 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:34123
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:34123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:45:39.308353 4016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:39.308691 4016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:39.308895 4016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:39.316546 4016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:45:39.316679 4016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:39.316773 4016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:39.316931 4016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:45:39.316987 4016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:45:39.328495 4016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:34123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:34123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:39.330052 4016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:39.331512 4016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:39.344882 4021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:39.344954 4022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:39.344856 4024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:39.345957 4016 server_base.cc:1034] running on GCE node
I20260806 19:45:39.347293 4016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:39.349363 4016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:39.350641 4016 hybrid_clock.cc:648] HybridClock initialized: now 1786045539350585 us; error 94 us; skew 500 ppm
I20260806 19:45:39.351117 4016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:39 dist-test-slave-77v9 krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045539, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:39.359022 4016 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:45:39.361339 4016 webserver.cc:460] Webserver started at http://127.21.202.126:44365/ using document root <none> and password file <none>
I20260806 19:45:39.362231 4016 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:39.362396 4016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:39.362731 4016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:45:39.366344 4016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "e776789a4e954d2f9118c36dee9e169a"
format_stamp: "Formatted at 2026-08-06 19:45:39 on dist-test-slave-77v9"
server_key: "172ac4ee59ab68bb23f93a874df130d6"
server_key_iv: "b09e59413308bed0a0f7d60855f3529d"
server_key_version: "encryptionkey@0"
I20260806 19:45:39.367242 4016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "e776789a4e954d2f9118c36dee9e169a"
format_stamp: "Formatted at 2026-08-06 19:45:39 on dist-test-slave-77v9"
server_key: "172ac4ee59ab68bb23f93a874df130d6"
server_key_iv: "b09e59413308bed0a0f7d60855f3529d"
server_key_version: "encryptionkey@0"
I20260806 19:45:39.372991 4016 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 19:45:39.376957 4031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:39.378669 4016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:45:39.378911 4016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "e776789a4e954d2f9118c36dee9e169a"
format_stamp: "Formatted at 2026-08-06 19:45:39 on dist-test-slave-77v9"
server_key: "172ac4ee59ab68bb23f93a874df130d6"
server_key_iv: "b09e59413308bed0a0f7d60855f3529d"
server_key_version: "encryptionkey@0"
I20260806 19:45:39.379146 4016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:39.404373 4016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:39.411315 4016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:39.411715 4016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:39.442198 4016 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:34123
I20260806 19:45:39.442248 4085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:34123 every 8 connection(s)
I20260806 19:45:39.443713 4016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:45:39.446357 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 4016
I20260806 19:45:39.446620 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:45:39.447397 22313 external_mini_cluster.cc:1428] Setting key 3d00eec47381429109d310ad67db1afc
I20260806 19:45:39.448957 4086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:39.467682 4086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Bootstrap starting.
Aug 06 19:45:39 dist-test-slave-77v9 krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:39.472119 4086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:39.473634 4086 log.cc:824] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:39.477054 4086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: No bootstrap required, opened a new log
I20260806 19:45:39.483873 4089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:39.450963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46522 (local address 127.21.202.126:34123)
0806 19:45:39.451893 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:39.451910 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:39.451968 (+ 58us) server_negotiation.cc:407] Connection header received
0806 19:45:39.454581 (+ 2613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:39.454618 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:39.455261 (+ 643us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:39.455820 (+ 559us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:39.457556 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:39.459862 (+ 2306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:39.462180 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:39.464189 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:39.464553 (+ 364us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:39.471214 (+ 6661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:39.471224 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:39.471241 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:39.471321 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:39.475291 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:39.476188 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:39.476196 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:39.476203 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:39.476388 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:39.477670 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:39.477689 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:39.477703 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:39.478961 (+ 1258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:39.480039 (+ 1078us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:39.480921 (+ 882us) server_negotiation.cc:299] Negotiation successful
0806 19:45:39.482266 (+ 1345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":408,"threads_started":1}
I20260806 19:45:39.489643 4086 raft_consensus.cc:348] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } }
I20260806 19:45:39.490053 4086 raft_consensus.cc:374] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:39.490121 4086 raft_consensus.cc:729] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e776789a4e954d2f9118c36dee9e169a, State: Initialized, Role: FOLLOWER
I20260806 19:45:39.491168 4086 consensus_queue.cc:260] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } }
I20260806 19:45:39.491493 4086 raft_consensus.cc:388] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:45:39.491668 4086 raft_consensus.cc:482] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:45:39.491896 4086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:39.494747 4086 raft_consensus.cc:504] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } }
I20260806 19:45:39.495879 4086 leader_election.cc:302] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e776789a4e954d2f9118c36dee9e169a; no voters:
I20260806 19:45:39.497282 4086 leader_election.cc:288] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:45:39.497344 4091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:45:39.498579 4091 raft_consensus.cc:686] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 1 LEADER]: Becoming Leader. State: Replica: e776789a4e954d2f9118c36dee9e169a, State: Running, Role: LEADER
I20260806 19:45:39.499295 4091 consensus_queue.cc:237] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } }
I20260806 19:45:39.500444 4086 sys_catalog.cc:564] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:45:39.502674 4092 sys_catalog.cc:455] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e776789a4e954d2f9118c36dee9e169a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } } }
I20260806 19:45:39.503029 4093 sys_catalog.cc:455] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e776789a4e954d2f9118c36dee9e169a. Latest consensus state: current_term: 1 leader_uuid: "e776789a4e954d2f9118c36dee9e169a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } } }
I20260806 19:45:39.503278 4092 sys_catalog.cc:458] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:39.503325 4093 sys_catalog.cc:458] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:39.505937 4097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:45:39.511804 4097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:45:39.521680 4097 catalog_manager.cc:1347] Generated new cluster ID: 555867a9580c4a968916979338e7b614
I20260806 19:45:39.521831 4097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:45:39.540771 4097 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:45:39.542199 4097 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:45:39.553435 4097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Generated new TSK 0
I20260806 19:45:39.554457 4097 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:45:39.636732 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34123
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:45:40.060029 4112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:40.060380 4112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:40.060483 4112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:40.070233 4112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:40.070403 4112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:40.070602 4112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:45:40.081902 4112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:40.083673 4112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:40.085294 4112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:40.102634 4117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:40.102526 4120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:40.102591 4118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:40.108100 4112 server_base.cc:1034] running on GCE node
I20260806 19:45:40.109093 4112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:40.110426 4112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:40.111974 4112 hybrid_clock.cc:648] HybridClock initialized: now 1786045540111830 us; error 91 us; skew 500 ppm
I20260806 19:45:40.112653 4112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:40 dist-test-slave-77v9 krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045540, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:40.117197 4112 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:45:40.119078 4112 webserver.cc:460] Webserver started at http://127.21.202.65:37065/ using document root <none> and password file <none>
I20260806 19:45:40.119984 4112 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:40.120148 4112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:40.120489 4112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:45:40.123829 4112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "6301b5c85d334595928148bee1ea2ddc"
format_stamp: "Formatted at 2026-08-06 19:45:40 on dist-test-slave-77v9"
server_key: "7054168269b467c145429271684e735b"
server_key_iv: "ddc65fa2849dac686fe56bb2fdbe79e1"
server_key_version: "encryptionkey@0"
I20260806 19:45:40.124725 4112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "6301b5c85d334595928148bee1ea2ddc"
format_stamp: "Formatted at 2026-08-06 19:45:40 on dist-test-slave-77v9"
server_key: "7054168269b467c145429271684e735b"
server_key_iv: "ddc65fa2849dac686fe56bb2fdbe79e1"
server_key_version: "encryptionkey@0"
I20260806 19:45:40.132170 4112 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260806 19:45:40.137038 4128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:40.138965 4112 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:45:40.139184 4112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "6301b5c85d334595928148bee1ea2ddc"
format_stamp: "Formatted at 2026-08-06 19:45:40 on dist-test-slave-77v9"
server_key: "7054168269b467c145429271684e735b"
server_key_iv: "ddc65fa2849dac686fe56bb2fdbe79e1"
server_key_version: "encryptionkey@0"
I20260806 19:45:40.139428 4112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:40.163918 4112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:40.168880 4112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:40.169210 4112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:40.170334 4112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:40.171811 4112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:40.171944 4112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:40.172101 4112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:40.172178 4112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:40.213524 4112 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:43791
I20260806 19:45:40.213589 4241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:43791 every 8 connection(s)
I20260806 19:45:40.215185 4112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:45:40.216459 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 4112
I20260806 19:45:40.216694 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:45:40.217621 22313 external_mini_cluster.cc:1428] Setting key 5a7e3ca8439e4deb6f68b85b42645971
Aug 06 19:45:40 dist-test-slave-77v9 krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045540, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:40.243446 4244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:40.219329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:38449 (local address 127.21.202.126:34123)
0806 19:45:40.219847 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:40.219854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:40.222249 (+ 2395us) server_negotiation.cc:407] Connection header received
0806 19:45:40.224285 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:40.224294 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:40.224405 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:40.224539 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:40.226493 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:40.227688 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:40.229605 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:40.230296 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:40.230462 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:40.234748 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:40.234758 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:40.234766 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:40.234816 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:40.239243 (+ 4427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:40.240301 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:40.240309 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:40.240315 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:40.240495 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:40.241133 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:40.241141 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:40.241147 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:40.241561 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:40.241745 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:40.242601 (+ 856us) server_negotiation.cc:299] Negotiation successful
0806 19:45:40.242916 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":274,"threads_started":1}
I20260806 19:45:40.244731 4245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:40.220481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:38449)
0806 19:45:40.221880 (+ 1399us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:40.221954 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:40.223962 (+ 2008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:40.224948 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:40.224960 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:40.225608 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:40.226267 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:40.226291 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:40.227935 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:40.227939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:40.229433 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:40.229449 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:40.230585 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:40.230591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:40.230810 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:40.231663 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:40.231689 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:40.234418 (+ 2729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:40.239506 (+ 5088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:40.239516 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:40.239533 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:40.240123 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:40.240633 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:40.240637 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:40.240639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:40.240998 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:40.241754 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:40.241762 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:40.242151 (+ 389us) client_negotiation.cc:769] Sending connection context
0806 19:45:40.242427 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 19:45:40.242866 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":489,"threads_started":1}
I20260806 19:45:40.247284 4242 heartbeater.cc:347] Connected to a master server at 127.21.202.126:34123
I20260806 19:45:40.247655 4242 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:40.248538 4242 heartbeater.cc:511] Master 127.21.202.126:34123 requested a full tablet report, sending...
I20260806 19:45:40.251634 4050 ts_manager.cc:194] Registered new tserver with Master: 6301b5c85d334595928148bee1ea2ddc (127.21.202.65:43791)
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
I20260806 19:45:40.253631 4050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:38449
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:45:40.297683 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34123
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:45:40.685928 4249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:40.686251 4249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:40.686470 4249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:40.694187 4249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:40.694340 4249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:40.694533 4249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:45:40.705252 4249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:40.706730 4249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:40.708228 4249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:40.724530 4254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:40.724622 4257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:40.724594 4255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:40.726300 4249 server_base.cc:1034] running on GCE node
I20260806 19:45:40.727046 4249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:40.728149 4249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:40.729563 4249 hybrid_clock.cc:648] HybridClock initialized: now 1786045540729432 us; error 114 us; skew 500 ppm
I20260806 19:45:40.729880 4249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:40 dist-test-slave-77v9 krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045540, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:40.733502 4249 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:45:40.735183 4249 webserver.cc:460] Webserver started at http://127.21.202.66:38499/ using document root <none> and password file <none>
I20260806 19:45:40.735890 4249 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:40.735997 4249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:40.736271 4249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:45:40.739383 4249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "beef8fd5b8d742489d9465e5ec6d8189"
format_stamp: "Formatted at 2026-08-06 19:45:40 on dist-test-slave-77v9"
server_key: "e4ca53ebed0bb6a09403a82b104f6858"
server_key_iv: "6887b4cf41263235d2bf8a463b58a194"
server_key_version: "encryptionkey@0"
I20260806 19:45:40.740140 4249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "beef8fd5b8d742489d9465e5ec6d8189"
format_stamp: "Formatted at 2026-08-06 19:45:40 on dist-test-slave-77v9"
server_key: "e4ca53ebed0bb6a09403a82b104f6858"
server_key_iv: "6887b4cf41263235d2bf8a463b58a194"
server_key_version: "encryptionkey@0"
I20260806 19:45:40.746877 4249 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 19:45:40.750854 4264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:40.752460 4249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:45:40.752642 4249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "beef8fd5b8d742489d9465e5ec6d8189"
format_stamp: "Formatted at 2026-08-06 19:45:40 on dist-test-slave-77v9"
server_key: "e4ca53ebed0bb6a09403a82b104f6858"
server_key_iv: "6887b4cf41263235d2bf8a463b58a194"
server_key_version: "encryptionkey@0"
I20260806 19:45:40.752919 4249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:40.780881 4249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:40.787668 4249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:40.787985 4249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:40.789069 4249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:40.790694 4249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:40.790778 4249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:40.790870 4249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:40.790906 4249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:40.833138 4249 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:46781
I20260806 19:45:40.833186 4380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:46781 every 8 connection(s)
I20260806 19:45:40.834474 4249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:45:40.843011 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 4249
I20260806 19:45:40.843328 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:45:40.843801 22313 external_mini_cluster.cc:1428] Setting key cee079c1c7219c8abe2982013a654272
Aug 06 19:45:40 dist-test-slave-77v9 krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045540, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:40.859601 4383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:40.837541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:55709 (local address 127.21.202.126:34123)
0806 19:45:40.838080 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:40.838085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:40.838855 (+ 770us) server_negotiation.cc:407] Connection header received
0806 19:45:40.840886 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:40.840902 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:40.840997 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:40.841129 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:40.843047 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:40.844029 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:40.846044 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:40.846698 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:40.846842 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:40.851208 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:40.851218 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:40.851226 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:40.851281 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:40.855157 (+ 3876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:40.856384 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:40.856393 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:40.856399 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:40.856577 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:40.857146 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:40.857154 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:40.857159 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:40.857636 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:40.857799 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:40.858802 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0806 19:45:40.859083 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":276,"threads_started":1}
I20260806 19:45:40.860649 4384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:40.837952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:55709)
0806 19:45:40.838678 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:40.838718 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:40.840512 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:40.841397 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:40.841406 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:40.841969 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:40.842693 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:40.842724 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:40.844288 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:40.844293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:40.845840 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:40.845860 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:40.846980 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:40.846987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:40.847263 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:40.848074 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:40.848100 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:40.850888 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:40.855387 (+ 4499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:40.855402 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:40.855427 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:40.856154 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:40.856715 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:40.856723 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:40.856729 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:40.857028 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:40.857798 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:40.857809 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:40.858335 (+ 526us) client_negotiation.cc:769] Sending connection context
0806 19:45:40.858651 (+ 316us) client_negotiation.cc:240] Negotiation successful
0806 19:45:40.859237 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":277,"threads_started":1}
I20260806 19:45:40.862663 4381 heartbeater.cc:347] Connected to a master server at 127.21.202.126:34123
I20260806 19:45:40.863262 4381 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:40.864166 4381 heartbeater.cc:511] Master 127.21.202.126:34123 requested a full tablet report, sending...
I20260806 19:45:40.866096 4050 ts_manager.cc:194] Registered new tserver with Master: beef8fd5b8d742489d9465e5ec6d8189 (127.21.202.66:46781)
I20260806 19:45:40.867085 4050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:55709
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:45:40.921026 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34123
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:45:41.257458 4242 heartbeater.cc:503] Master 127.21.202.126:34123 was elected leader, sending a full tablet report...
W20260806 19:45:41.302390 4387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:41.302703 4387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:41.302874 4387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:41.310822 4387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:41.310957 4387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:41.311079 4387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:45:41.322971 4387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:34123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:41.324441 4387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:41.325994 4387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:41.341073 4392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:41.341148 4393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:41.341109 4395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:41.342084 4387 server_base.cc:1034] running on GCE node
I20260806 19:45:41.342882 4387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:41.344233 4387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:41.345496 4387 hybrid_clock.cc:648] HybridClock initialized: now 1786045541345435 us; error 42 us; skew 500 ppm
I20260806 19:45:41.345853 4387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:41 dist-test-slave-77v9 krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045541, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:41.349726 4387 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:45:41.351686 4387 webserver.cc:460] Webserver started at http://127.21.202.67:39399/ using document root <none> and password file <none>
I20260806 19:45:41.352473 4387 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:41.352598 4387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:41.353024 4387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:45:41.356617 4387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "26160dc9a0fb4c2db9111aad5e9a02bb"
format_stamp: "Formatted at 2026-08-06 19:45:41 on dist-test-slave-77v9"
server_key: "4bb607f8f35ec1c3e7949ddd4ca3f09d"
server_key_iv: "da1b7f452a514f18c09b1ca01afbf5af"
server_key_version: "encryptionkey@0"
I20260806 19:45:41.357774 4387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "26160dc9a0fb4c2db9111aad5e9a02bb"
format_stamp: "Formatted at 2026-08-06 19:45:41 on dist-test-slave-77v9"
server_key: "4bb607f8f35ec1c3e7949ddd4ca3f09d"
server_key_iv: "da1b7f452a514f18c09b1ca01afbf5af"
server_key_version: "encryptionkey@0"
I20260806 19:45:41.364728 4387 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:45:41.369396 4403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:41.371140 4387 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:45:41.371376 4387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "26160dc9a0fb4c2db9111aad5e9a02bb"
format_stamp: "Formatted at 2026-08-06 19:45:41 on dist-test-slave-77v9"
server_key: "4bb607f8f35ec1c3e7949ddd4ca3f09d"
server_key_iv: "da1b7f452a514f18c09b1ca01afbf5af"
server_key_version: "encryptionkey@0"
I20260806 19:45:41.371632 4387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:41.389117 4387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:41.394222 4387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:41.394560 4387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:41.395675 4387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:41.397253 4387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:41.397363 4387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:41.397523 4387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:41.397604 4387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:41.442559 4387 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:36971
I20260806 19:45:41.442571 4519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:36971 every 8 connection(s)
I20260806 19:45:41.444522 4387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:45:41.445827 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 4387
I20260806 19:45:41.446118 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:45:41.447047 22313 external_mini_cluster.cc:1428] Setting key 619c2dd2d974ebe9cdbeb7f76689dab7
Aug 06 19:45:41 dist-test-slave-77v9 krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045541, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:41.478507 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:41.449560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:55613 (local address 127.21.202.126:34123)
0806 19:45:41.457947 (+ 8387us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:41.457952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:41.458397 (+ 445us) server_negotiation.cc:407] Connection header received
0806 19:45:41.460410 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:41.460416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:41.460508 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:41.460656 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:41.462364 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.463197 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.465460 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.465909 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.466055 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:41.470365 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:41.470370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:41.470373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:41.470416 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:41.474191 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:41.475169 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:41.475175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:41.475179 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:41.475336 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:41.475843 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:41.475848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:41.475852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:41.476155 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:41.476336 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:41.477333 (+ 997us) server_negotiation.cc:299] Negotiation successful
0806 19:45:41.477891 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8266,"thread_start_us":8085,"threads_started":1}
I20260806 19:45:41.480018 4522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:41.448909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:55613)
0806 19:45:41.458209 (+ 9300us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:41.458264 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:41.460091 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:41.460966 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:41.460975 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:41.461529 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:41.462181 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:41.462200 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.463386 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.463390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:41.465304 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:41.465319 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.466155 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.466165 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:41.466498 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:41.467479 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:41.467506 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:41.470080 (+ 2574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:41.474413 (+ 4333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:41.474426 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:41.474448 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:41.474995 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:41.475480 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:41.475485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:41.475489 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:41.475706 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:41.476317 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:41.476325 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:41.476867 (+ 542us) client_negotiation.cc:769] Sending connection context
0806 19:45:41.477216 (+ 349us) client_negotiation.cc:240] Negotiation successful
0806 19:45:41.478320 (+ 1104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8420,"thread_start_us":8110,"threads_started":1}
I20260806 19:45:41.481868 4520 heartbeater.cc:347] Connected to a master server at 127.21.202.126:34123
I20260806 19:45:41.482285 4520 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:41.483278 4520 heartbeater.cc:511] Master 127.21.202.126:34123 requested a full tablet report, sending...
I20260806 19:45:41.485216 4050 ts_manager.cc:194] Registered new tserver with Master: 26160dc9a0fb4c2db9111aad5e9a02bb (127.21.202.67:36971)
I20260806 19:45:41.486109 4050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:55613
I20260806 19:45:41.491623 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:45:41.514012 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:41.496658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46530 (local address 127.21.202.126:34123)
0806 19:45:41.496952 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:41.496957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:41.497326 (+ 369us) server_negotiation.cc:407] Connection header received
0806 19:45:41.497489 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:41.497494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:41.497609 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:41.497707 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:41.499318 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.500097 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.501874 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.502425 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.502540 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:41.505487 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:41.505492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:41.505496 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:41.505538 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:41.510159 (+ 4621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:41.511075 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:41.511080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:41.511082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:41.511234 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:41.511805 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:41.511810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:41.511812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:41.512245 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:41.512478 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:41.513400 (+ 922us) server_negotiation.cc:299] Negotiation successful
0806 19:45:41.513631 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 19:45:41.521270 4050 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:45:41.529115 4050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:45:41.577253 4536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:41.549054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40704 (local address 127.21.202.66:46781)
0806 19:45:41.564603 (+ 15549us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:41.564617 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:41.566589 (+ 1972us) server_negotiation.cc:407] Connection header received
0806 19:45:41.566710 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:41.566718 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:41.567063 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:41.567789 (+ 726us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:41.569113 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.570809 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.573702 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.575183 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.575338 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:41.575777 (+ 439us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:41.576590 (+ 813us) server_negotiation.cc:299] Negotiation successful
0806 19:45:41.576846 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15291,"thread_start_us":15042,"threads_started":1}
I20260806 19:45:41.577335 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:41.555858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52138 (local address 127.21.202.65:43791)
0806 19:45:41.561660 (+ 5802us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:41.561674 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:41.566690 (+ 5016us) server_negotiation.cc:407] Connection header received
0806 19:45:41.566980 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:41.566988 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:41.567199 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:41.567369 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:41.569112 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.570812 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.574426 (+ 3614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.575699 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.575988 (+ 289us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:41.576234 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:41.576725 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 19:45:41.576987 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5537,"thread_start_us":5272,"threads_started":1}
I20260806 19:45:41.578083 4537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:41.548201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:36971 (local address 127.0.0.1:36614)
0806 19:45:41.565078 (+ 16877us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:41.565126 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:41.565329 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:41.568852 (+ 3523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:41.568858 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:41.568888 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:41.569304 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:41.569314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.570807 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.570815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:41.574128 (+ 3313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:41.574144 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.576363 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.576369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:41.576554 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:41.576583 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 19:45:41.576639 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 19:45:41.576745 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16582,"mutex_wait_us":134,"thread_start_us":17685,"threads_started":1}
I20260806 19:45:41.578197 4533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:41.548426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:46781 (local address 127.0.0.1:40704)
0806 19:45:41.566206 (+ 17780us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:41.566226 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:41.566421 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:41.567351 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:41.567358 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:41.567567 (+ 209us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:41.568905 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:41.568919 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.570960 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.570972 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:41.573483 (+ 2511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:41.573498 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.575956 (+ 2458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.575965 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:41.576237 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:41.576342 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 19:45:41.576480 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 19:45:41.576697 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17618,"mutex_wait_us":32,"thread_start_us":18996,"threads_started":1}
I20260806 19:45:41.579049 4534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:41.555153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:43791 (local address 127.0.0.1:52138)
0806 19:45:41.566564 (+ 11411us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:41.566582 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:41.566786 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:41.567731 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:41.567736 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:41.567759 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:41.568730 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:41.568762 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.571331 (+ 2569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.571336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:41.574128 (+ 2792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:41.574144 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.576042 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.576050 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:41.576286 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:41.576473 (+ 187us) client_negotiation.cc:769] Sending connection context
0806 19:45:41.577572 (+ 1099us) client_negotiation.cc:240] Negotiation successful
0806 19:45:41.577732 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11249,"mutex_wait_us":180,"thread_start_us":9433,"threads_started":1}
I20260806 19:45:41.580634 4535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:41.548563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36614 (local address 127.21.202.67:36971)
0806 19:45:41.568264 (+ 19701us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:41.568274 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:41.568317 (+ 43us) server_negotiation.cc:407] Connection header received
0806 19:45:41.568396 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:41.568402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:41.568596 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:41.568809 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:41.569499 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.570640 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.574372 (+ 3732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.576078 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.579259 (+ 3181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:41.579532 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:41.579610 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 19:45:41.579923 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19502,"thread_start_us":19311,"threads_started":1}
I20260806 19:45:41.583833 4313 tablet_service.cc:1467] Processing CreateTablet for tablet 19a4eee7906340f8b2ccf7288f42d6a7 (DEFAULT_TABLE table=test-table [id=e4213f68707f40758ac41b81aa0669ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:45:41.584573 4176 tablet_service.cc:1467] Processing CreateTablet for tablet 19a4eee7906340f8b2ccf7288f42d6a7 (DEFAULT_TABLE table=test-table [id=e4213f68707f40758ac41b81aa0669ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:45:41.585397 4313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19a4eee7906340f8b2ccf7288f42d6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:41.586025 4176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19a4eee7906340f8b2ccf7288f42d6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:41.586055 4452 tablet_service.cc:1467] Processing CreateTablet for tablet 19a4eee7906340f8b2ccf7288f42d6a7 (DEFAULT_TABLE table=test-table [id=e4213f68707f40758ac41b81aa0669ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:45:41.587494 4452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19a4eee7906340f8b2ccf7288f42d6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:41.599720 4539 tablet_bootstrap.cc:492] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189: Bootstrap starting.
I20260806 19:45:41.602617 4540 tablet_bootstrap.cc:492] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc: Bootstrap starting.
I20260806 19:45:41.604132 4539 tablet_bootstrap.cc:654] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:41.605919 4539 log.cc:824] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:41.606995 4540 tablet_bootstrap.cc:654] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:41.609581 4540 log.cc:824] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:41.611485 4539 tablet_bootstrap.cc:492] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189: No bootstrap required, opened a new log
I20260806 19:45:41.611994 4539 ts_tablet_manager.cc:1397] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260806 19:45:41.612098 4542 tablet_bootstrap.cc:492] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb: Bootstrap starting.
I20260806 19:45:41.614699 4540 tablet_bootstrap.cc:492] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc: No bootstrap required, opened a new log
I20260806 19:45:41.615134 4540 ts_tablet_manager.cc:1397] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260806 19:45:41.616155 4542 tablet_bootstrap.cc:654] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:41.617961 4542 log.cc:824] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:41.622339 4542 tablet_bootstrap.cc:492] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb: No bootstrap required, opened a new log
I20260806 19:45:41.622921 4542 ts_tablet_manager.cc:1397] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 19:45:41.627087 4539 raft_consensus.cc:348] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } } peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } }
I20260806 19:45:41.627640 4539 raft_consensus.cc:374] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:41.627815 4539 raft_consensus.cc:729] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beef8fd5b8d742489d9465e5ec6d8189, State: Initialized, Role: FOLLOWER
I20260806 19:45:41.628746 4539 consensus_queue.cc:260] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } } peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } }
I20260806 19:45:41.630187 4381 heartbeater.cc:503] Master 127.21.202.126:34123 was elected leader, sending a full tablet report...
I20260806 19:45:41.629945 4540 raft_consensus.cc:348] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } } peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } }
I20260806 19:45:41.630402 4540 raft_consensus.cc:374] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:41.630505 4540 raft_consensus.cc:729] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6301b5c85d334595928148bee1ea2ddc, State: Initialized, Role: FOLLOWER
I20260806 19:45:41.631036 4539 ts_tablet_manager.cc:1428] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189: Time spent starting tablet: real 0.019s user 0.018s sys 0.000s
I20260806 19:45:41.631330 4540 consensus_queue.cc:260] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } } peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } }
I20260806 19:45:41.632058 4542 raft_consensus.cc:348] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } } peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } }
I20260806 19:45:41.632581 4542 raft_consensus.cc:374] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:41.632704 4542 raft_consensus.cc:729] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26160dc9a0fb4c2db9111aad5e9a02bb, State: Initialized, Role: FOLLOWER
I20260806 19:45:41.633507 4542 consensus_queue.cc:260] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } } peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } }
I20260806 19:45:41.634625 4540 ts_tablet_manager.cc:1428] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc: Time spent starting tablet: real 0.019s user 0.017s sys 0.000s
I20260806 19:45:41.636010 4520 heartbeater.cc:503] Master 127.21.202.126:34123 was elected leader, sending a full tablet report...
I20260806 19:45:41.636696 4542 ts_tablet_manager.cc:1428] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
W20260806 19:45:41.702940 4521 tablet.cc:2369] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:45:41.719777 4243 tablet.cc:2369] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:45:41.837013 4382 tablet.cc:2369] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:45:41.947783 4547 raft_consensus.cc:482] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:45:41.948177 4547 raft_consensus.cc:504] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } } peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } }
I20260806 19:45:41.950037 4547 leader_election.cc:288] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers beef8fd5b8d742489d9465e5ec6d8189 (127.21.202.66:46781), 6301b5c85d334595928148bee1ea2ddc (127.21.202.65:43791)
I20260806 19:45:41.957300 4522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:41.950351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:43791 (local address 127.21.202.67:46453)
0806 19:45:41.950665 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:41.950687 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:41.950822 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:41.951275 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:41.951280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:41.951309 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:41.951665 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:41.951678 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.953041 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.953048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:41.954817 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:41.954830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.956277 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.956285 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:41.956502 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:41.956540 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 19:45:41.956684 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 19:45:41.956885 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"spinlock_wait_cycles":104704}
I20260806 19:45:41.957345 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:41.950585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:46453 (local address 127.21.202.65:43791)
0806 19:45:41.950797 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:41.950803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:41.950826 (+ 23us) server_negotiation.cc:407] Connection header received
0806 19:45:41.951051 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:41.951059 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:41.951140 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:41.951238 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:41.951877 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.952908 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.954980 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.956102 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.956226 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:41.956284 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:41.956854 (+ 570us) server_negotiation.cc:299] Negotiation successful
0806 19:45:41.957018 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 19:45:41.958503 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:41.950546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:46781 (local address 127.21.202.67:43297)
0806 19:45:41.951288 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:41.951307 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:41.951454 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:41.951871 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:41.951876 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:41.951903 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:41.952318 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:41.952330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.953736 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.953741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:41.955616 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:41.955631 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.957649 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.957658 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:41.957901 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:41.957940 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 19:45:41.958063 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 19:45:41.958223 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":420,"threads_started":1}
I20260806 19:45:41.958662 4536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:41.950631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:43297 (local address 127.21.202.66:46781)
0806 19:45:41.950913 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:41.950923 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:41.951402 (+ 479us) server_negotiation.cc:407] Connection header received
0806 19:45:41.951590 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:41.951600 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:41.951684 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:41.951796 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:41.952485 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.953560 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.955941 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:41.957342 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:41.957524 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:41.957616 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:41.958136 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 19:45:41.958372 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 19:45:41.958882 4197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a4eee7906340f8b2ccf7288f42d6a7" candidate_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6301b5c85d334595928148bee1ea2ddc" is_pre_election: true
I20260806 19:45:41.959506 4197 raft_consensus.cc:2454] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26160dc9a0fb4c2db9111aad5e9a02bb in term 0.
I20260806 19:45:41.959780 4334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a4eee7906340f8b2ccf7288f42d6a7" candidate_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beef8fd5b8d742489d9465e5ec6d8189" is_pre_election: true
I20260806 19:45:41.960366 4334 raft_consensus.cc:2454] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26160dc9a0fb4c2db9111aad5e9a02bb in term 0.
I20260806 19:45:41.960381 4405 leader_election.cc:302] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26160dc9a0fb4c2db9111aad5e9a02bb, 6301b5c85d334595928148bee1ea2ddc; no voters:
I20260806 19:45:41.960794 4547 raft_consensus.cc:2781] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:45:41.961030 4547 raft_consensus.cc:482] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:45:41.961176 4547 raft_consensus.cc:3037] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:41.963908 4547 raft_consensus.cc:504] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } } peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } }
I20260806 19:45:41.964699 4547 leader_election.cc:288] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [CANDIDATE]: Term 1 election: Requested vote from peers beef8fd5b8d742489d9465e5ec6d8189 (127.21.202.66:46781), 6301b5c85d334595928148bee1ea2ddc (127.21.202.65:43791)
I20260806 19:45:41.965299 4334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a4eee7906340f8b2ccf7288f42d6a7" candidate_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beef8fd5b8d742489d9465e5ec6d8189"
I20260806 19:45:41.965365 4197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a4eee7906340f8b2ccf7288f42d6a7" candidate_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6301b5c85d334595928148bee1ea2ddc"
I20260806 19:45:41.965555 4334 raft_consensus.cc:3037] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:41.965597 4197 raft_consensus.cc:3037] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:41.968094 4334 raft_consensus.cc:2454] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26160dc9a0fb4c2db9111aad5e9a02bb in term 1.
I20260806 19:45:41.968197 4197 raft_consensus.cc:2454] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26160dc9a0fb4c2db9111aad5e9a02bb in term 1.
I20260806 19:45:41.968776 4404 leader_election.cc:302] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26160dc9a0fb4c2db9111aad5e9a02bb, beef8fd5b8d742489d9465e5ec6d8189; no voters:
I20260806 19:45:41.969221 4547 raft_consensus.cc:2781] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:45:41.969696 4547 raft_consensus.cc:686] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 1 LEADER]: Becoming Leader. State: Replica: 26160dc9a0fb4c2db9111aad5e9a02bb, State: Running, Role: LEADER
I20260806 19:45:41.970152 4547 consensus_queue.cc:237] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } } peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } }
I20260806 19:45:41.976410 4049 catalog_manager.cc:5495] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 26160dc9a0fb4c2db9111aad5e9a02bb (127.21.202.67). New cstate: current_term: 1 leader_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } health_report { overall_health: UNKNOWN } } }
I20260806 19:45:42.029426 4535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:42.022532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36630 (local address 127.21.202.67:36971)
0806 19:45:42.022729 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:42.022734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:42.023130 (+ 396us) server_negotiation.cc:407] Connection header received
0806 19:45:42.023314 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:42.023320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:42.023404 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:42.023553 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:45:42.024081 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:42.024955 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:42.026673 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:42.027283 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:42.027412 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:42.028086 (+ 674us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:45:42.028607 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:45:42.028726 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:42.029092 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 19:45:42.029174 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 19:45:42.039702 4197 raft_consensus.cc:1261] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc [term 1 FOLLOWER]: Refusing update from remote peer 26160dc9a0fb4c2db9111aad5e9a02bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:45:42.039702 4334 raft_consensus.cc:1261] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189 [term 1 FOLLOWER]: Refusing update from remote peer 26160dc9a0fb4c2db9111aad5e9a02bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:45:42.041069 4549 consensus_queue.cc:1035] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:45:42.041495 4547 consensus_queue.cc:1035] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:45:42.073237 4050 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:45:42.073588 4050 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:45:42.078807 4050 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Sending DeleteTablet for 3 replicas of tablet 19a4eee7906340f8b2ccf7288f42d6a7
I20260806 19:45:42.080636 4176 tablet_service.cc:1514] Processing DeleteTablet for tablet 19a4eee7906340f8b2ccf7288f42d6a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:45:42 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:52138
I20260806 19:45:42.081564 4313 tablet_service.cc:1514] Processing DeleteTablet for tablet 19a4eee7906340f8b2ccf7288f42d6a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:45:42 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:40704
I20260806 19:45:42.081781 4452 tablet_service.cc:1514] Processing DeleteTablet for tablet 19a4eee7906340f8b2ccf7288f42d6a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:45:42 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:36614
I20260806 19:45:42.083043 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 4112
I20260806 19:45:42.092730 4566 tablet_replica.cc:330] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189: stopping tablet replica
I20260806 19:45:42.093688 4566 raft_consensus.cc:2229] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:45:42.094830 4566 raft_consensus.cc:2258] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:45:42.095774 4565 tablet_replica.cc:330] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb: stopping tablet replica
I20260806 19:45:42.096110 4564 tablet_replica.cc:330] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc: stopping tablet replica
I20260806 19:45:42.096331 4565 raft_consensus.cc:2229] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:45:42.096582 4564 raft_consensus.cc:2229] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:45:42.097117 4565 raft_consensus.cc:2258] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:45:42.097321 4564 raft_consensus.cc:2258] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:45:42.099735 4566 ts_tablet_manager.cc:1905] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:45:42.103580 4564 ts_tablet_manager.cc:1905] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:45:42.105870 4565 ts_tablet_manager.cc:1905] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:45:42.110399 4566 ts_tablet_manager.cc:1918] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:45:42.110730 4566 log.cc:1196] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/19a4eee7906340f8b2ccf7288f42d6a7
I20260806 19:45:42.111492 4566 ts_tablet_manager.cc:1939] T 19a4eee7906340f8b2ccf7288f42d6a7 P beef8fd5b8d742489d9465e5ec6d8189: Deleting consensus metadata
Aug 06 19:45:42 dist-test-slave-77v9 krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:45:42.113983 4032 catalog_manager.cc:4841] TS beef8fd5b8d742489d9465e5ec6d8189 (127.21.202.66:46781): tablet 19a4eee7906340f8b2ccf7288f42d6a7 (table test-table [id=e4213f68707f40758ac41b81aa0669ff]) successfully deleted
I20260806 19:45:42.116930 4564 ts_tablet_manager.cc:1918] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:45:42.117215 4564 log.cc:1196] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/19a4eee7906340f8b2ccf7288f42d6a7
I20260806 19:45:42.117866 4564 ts_tablet_manager.cc:1939] T 19a4eee7906340f8b2ccf7288f42d6a7 P 6301b5c85d334595928148bee1ea2ddc: Deleting consensus metadata
I20260806 19:45:42.119779 4565 ts_tablet_manager.cc:1918] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:45:42.120033 4565 log.cc:1196] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/19a4eee7906340f8b2ccf7288f42d6a7
I20260806 19:45:42.119992 4033 catalog_manager.cc:4841] TS 6301b5c85d334595928148bee1ea2ddc (127.21.202.65:43791): tablet 19a4eee7906340f8b2ccf7288f42d6a7 (table test-table [id=e4213f68707f40758ac41b81aa0669ff]) successfully deleted
I20260806 19:45:42.120733 4565 ts_tablet_manager.cc:1939] T 19a4eee7906340f8b2ccf7288f42d6a7 P 26160dc9a0fb4c2db9111aad5e9a02bb: Deleting consensus metadata
I20260806 19:45:42.122309 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:42.084985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52148 (local address 127.21.202.65:43791)
0806 19:45:42.085240 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:42.085250 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:42.098564 (+ 13314us) server_negotiation.cc:407] Connection header received
0806 19:45:42.098867 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:42.098874 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:42.098967 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:42.099050 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:42.101134 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:42.102968 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:42.105785 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:42.106582 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:42.106739 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:42.113710 (+ 6971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:42.113728 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:42.113750 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:42.113815 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:42.118275 (+ 4460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:42.119356 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:42.119365 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:42.119368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:42.119565 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:42.120212 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:42.120219 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:42.120223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:42.120735 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:42.121178 (+ 443us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:42.121534 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 19:45:42.121769 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 19:45:42.122731 4035 catalog_manager.cc:4841] TS 26160dc9a0fb4c2db9111aad5e9a02bb (127.21.202.67:36971): tablet 19a4eee7906340f8b2ccf7288f42d6a7 (table test-table [id=e4213f68707f40758ac41b81aa0669ff]) successfully deleted
I20260806 19:45:42.123104 4237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:42.248131 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 4112
I20260806 19:45:42.268935 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 4249
Aug 06 19:45:42 dist-test-slave-77v9 krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:45:42.289142 4536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:42.269937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40718 (local address 127.21.202.66:46781)
0806 19:45:42.270143 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:42.270150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:42.270174 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:45:42.270354 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:42.270361 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:42.270457 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:42.270584 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:42.272131 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:42.273383 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:42.274824 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:42.275698 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:42.275855 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:42.280987 (+ 5132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:42.281010 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:42.281035 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:42.281096 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:42.285572 (+ 4476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:42.286486 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:42.286498 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:42.286501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:42.286679 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:42.287136 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:42.287142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:42.287146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:42.287668 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:42.287976 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:42.288355 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 19:45:42.288710 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 19:45:42.290082 4376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:42.390743 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 4249
I20260806 19:45:42.414124 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 4387
Aug 06 19:45:42 dist-test-slave-77v9 krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:45:42.433918 4535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:42.415222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.21.202.67:36971)
0806 19:45:42.415430 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:42.415434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:42.415455 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:45:42.415686 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:42.415691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:42.415761 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:42.415865 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:42.417796 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:42.418647 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:42.420578 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:42.421157 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:42.421273 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:42.427299 (+ 6026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:42.427315 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:42.427333 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:42.427389 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:42.430773 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:42.431567 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:42.431584 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:42.431586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:42.431739 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:42.432137 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:42.432142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:42.432146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:42.432597 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:42.433093 (+ 496us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:42.433253 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 19:45:42.433498 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260806 19:45:42.434765 4515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:42.571370 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 4387
I20260806 19:45:42.595932 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 4016
I20260806 19:45:42.597316 4081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:42.720039 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 4016
I20260806 19:45:42.748900 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:34123
--webserver_interface=127.21.202.126
--webserver_port=44365
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:34123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:45:43.136512 4572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:43.136941 4572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:43.137058 4572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:43.145052 4572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:45:43.145182 4572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:43.145231 4572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:43.145282 4572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:45:43.145328 4572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:45:43.156252 4572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:34123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:34123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=44365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:43.157713 4572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:43.159142 4572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:43.185925 4578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:43.185981 4579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:43.186014 4581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:43.187129 4572 server_base.cc:1034] running on GCE node
I20260806 19:45:43.188102 4572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:43.189898 4572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:43.191146 4572 hybrid_clock.cc:648] HybridClock initialized: now 1786045543191082 us; error 49 us; skew 500 ppm
I20260806 19:45:43.191605 4572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:43 dist-test-slave-77v9 krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045543, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:43.196091 4572 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:45:43.198324 4572 webserver.cc:460] Webserver started at http://127.21.202.126:44365/ using document root <none> and password file <none>
I20260806 19:45:43.199223 4572 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:43.199404 4572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:43.206418 4572 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 19:45:43.210000 4588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:43.211612 4572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:45:43.211822 4572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "e776789a4e954d2f9118c36dee9e169a"
format_stamp: "Formatted at 2026-08-06 19:45:39 on dist-test-slave-77v9"
server_key: "172ac4ee59ab68bb23f93a874df130d6"
server_key_iv: "b09e59413308bed0a0f7d60855f3529d"
server_key_version: "encryptionkey@0"
I20260806 19:45:43.213107 4572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:43.226329 4572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:43.231775 4572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:43.232124 4572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:43.254981 4572 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:34123
I20260806 19:45:43.255075 4640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:34123 every 8 connection(s)
I20260806 19:45:43.256686 4572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:45:43.265333 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 4572
I20260806 19:45:43.265610 4641 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:45:43.266418 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:43791
--local_ip_for_outbound_sockets=127.21.202.65
--tserver_master_addrs=127.21.202.126:34123
--webserver_port=37065
--webserver_interface=127.21.202.65
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:45:43.270272 4641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Bootstrap starting.
I20260806 19:45:43.294517 4641 log.cc:824] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:43.311764 4641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:45:43.312413 4641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Bootstrap complete.
I20260806 19:45:43.325088 4641 raft_consensus.cc:348] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } }
I20260806 19:45:43.326233 4641 raft_consensus.cc:729] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e776789a4e954d2f9118c36dee9e169a, State: Initialized, Role: FOLLOWER
I20260806 19:45:43.326998 4641 consensus_queue.cc:260] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } }
I20260806 19:45:43.327325 4641 raft_consensus.cc:388] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:45:43.327489 4641 raft_consensus.cc:482] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:45:43.327890 4641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 1 FOLLOWER]: Advancing to term 2
I20260806 19:45:43.332600 4641 raft_consensus.cc:504] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } }
I20260806 19:45:43.334050 4641 leader_election.cc:302] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e776789a4e954d2f9118c36dee9e169a; no voters:
I20260806 19:45:43.335270 4641 leader_election.cc:288] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 19:45:43.335661 4645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 2 FOLLOWER]: Leader election won for term 2
I20260806 19:45:43.337167 4645 raft_consensus.cc:686] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 2 LEADER]: Becoming Leader. State: Replica: e776789a4e954d2f9118c36dee9e169a, State: Running, Role: LEADER
I20260806 19:45:43.337931 4645 consensus_queue.cc:237] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } }
I20260806 19:45:43.338925 4641 sys_catalog.cc:564] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:45:43.341378 4647 sys_catalog.cc:455] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e776789a4e954d2f9118c36dee9e169a. Latest consensus state: current_term: 2 leader_uuid: "e776789a4e954d2f9118c36dee9e169a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } } }
I20260806 19:45:43.341823 4647 sys_catalog.cc:458] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:43.342451 4646 sys_catalog.cc:455] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e776789a4e954d2f9118c36dee9e169a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } } }
I20260806 19:45:43.342738 4646 sys_catalog.cc:458] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:43.353073 4652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:45:43.364560 4652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:45:43.367959 4652 catalog_manager.cc:1259] Loaded cluster ID: 555867a9580c4a968916979338e7b614
I20260806 19:45:43.368247 4652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:45:43.373126 4652 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 19:45:43.399600 4652 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 19:45:43.409426 4652 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:45:43.411345 4652 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Loaded TSK: 0
I20260806 19:45:43.413061 4652 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 19:45:43.673075 4643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:43.673506 4643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:43.673695 4643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:43.683710 4643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:43.684003 4643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:43.684536 4643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:45:43.696549 4643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:43791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=37065
--tserver_master_addrs=127.21.202.126:34123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:43.698045 4643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:43.699713 4643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:43.717180 4672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:43.717154 4670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:43.718930 4643 server_base.cc:1034] running on GCE node
W20260806 19:45:43.720928 4669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:43.721904 4643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:43.723321 4643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:43.726894 4643 hybrid_clock.cc:648] HybridClock initialized: now 1786045543726758 us; error 80 us; skew 500 ppm
I20260806 19:45:43.727577 4643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:43 dist-test-slave-77v9 krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045543, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:43.732264 4643 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:45:43.734311 4643 webserver.cc:460] Webserver started at http://127.21.202.65:37065/ using document root <none> and password file <none>
I20260806 19:45:43.735160 4643 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:43.735328 4643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:43.743768 4643 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:45:43.747360 4679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:43.749269 4643 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:45:43.749478 4643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "6301b5c85d334595928148bee1ea2ddc"
format_stamp: "Formatted at 2026-08-06 19:45:40 on dist-test-slave-77v9"
server_key: "7054168269b467c145429271684e735b"
server_key_iv: "ddc65fa2849dac686fe56bb2fdbe79e1"
server_key_version: "encryptionkey@0"
I20260806 19:45:43.750567 4643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:43.783264 4643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:43.790866 4643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:43.791234 4643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:43.792595 4643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:43.794483 4643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:43.794624 4643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:43.794713 4643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:43.794771 4643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:43.836143 4643 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:43791
I20260806 19:45:43.836201 4792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:43791 every 8 connection(s)
I20260806 19:45:43.837918 4643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:45:43.845891 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 4643
I20260806 19:45:43.846625 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:46781
--local_ip_for_outbound_sockets=127.21.202.66
--tserver_master_addrs=127.21.202.126:34123
--webserver_port=38499
--webserver_interface=127.21.202.66
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:45:43 dist-test-slave-77v9 krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045543, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:43.868779 4796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:43.841910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:51999)
0806 19:45:43.842728 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:43.842773 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:43.844997 (+ 2224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:43.846686 (+ 1689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:43.846698 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:43.847404 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:43.848015 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:43.848035 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:43.850230 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:43.850234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:43.851880 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:43.851896 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:43.853203 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:43.853213 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:43.853537 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:43.854506 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:43.854544 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:43.857768 (+ 3224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:43.863599 (+ 5831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:43.863608 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:43.863622 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:43.864205 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:43.865150 (+ 945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:43.865154 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:43.865157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:43.865372 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:43.866531 (+ 1159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:43.866538 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:43.866890 (+ 352us) client_negotiation.cc:769] Sending connection context
0806 19:45:43.867157 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 19:45:43.867629 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":352,"threads_started":1}
I20260806 19:45:43.869270 4795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:43.841603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:51999 (local address 127.21.202.126:34123)
0806 19:45:43.842211 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:43.842257 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:43.842937 (+ 680us) server_negotiation.cc:407] Connection header received
0806 19:45:43.845602 (+ 2665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:43.845616 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:43.846066 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:43.846501 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:43.848249 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:43.850018 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:43.852101 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:43.852921 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:43.853144 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:43.858154 (+ 5010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:43.858163 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:43.858183 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:43.858231 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:43.863314 (+ 5083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:43.864404 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:43.864418 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:43.864425 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:43.864695 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:43.865546 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:43.865555 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:43.865561 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:43.866321 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:43.866543 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:43.867448 (+ 905us) server_negotiation.cc:299] Negotiation successful
0806 19:45:43.867933 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":274,"threads_started":1}
I20260806 19:45:43.873282 4793 heartbeater.cc:347] Connected to a master server at 127.21.202.126:34123
I20260806 19:45:43.873683 4793 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:43.874853 4793 heartbeater.cc:511] Master 127.21.202.126:34123 requested a full tablet report, sending...
I20260806 19:45:43.878679 4605 ts_manager.cc:194] Registered new tserver with Master: 6301b5c85d334595928148bee1ea2ddc (127.21.202.65:43791)
I20260806 19:45:43.881264 4605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:51999
W20260806 19:45:44.244580 4797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:44.244992 4797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:44.245105 4797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:44.253015 4797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:44.253149 4797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:44.253273 4797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:45:44.268913 4797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:46781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=38499
--tserver_master_addrs=127.21.202.126:34123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:44.270490 4797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:44.271981 4797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:44.288057 4802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:44.288028 4805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:44.288105 4803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:44.289167 4797 server_base.cc:1034] running on GCE node
I20260806 19:45:44.289921 4797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:44.291093 4797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:44.292435 4797 hybrid_clock.cc:648] HybridClock initialized: now 1786045544292378 us; error 40 us; skew 500 ppm
I20260806 19:45:44.293093 4797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:44 dist-test-slave-77v9 krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045544, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:44.297493 4797 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:45:44.299687 4797 webserver.cc:460] Webserver started at http://127.21.202.66:38499/ using document root <none> and password file <none>
I20260806 19:45:44.300433 4797 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:44.300567 4797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:44.306988 4797 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 19:45:44.311206 4812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:44.313228 4797 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:45:44.313459 4797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "beef8fd5b8d742489d9465e5ec6d8189"
format_stamp: "Formatted at 2026-08-06 19:45:40 on dist-test-slave-77v9"
server_key: "e4ca53ebed0bb6a09403a82b104f6858"
server_key_iv: "6887b4cf41263235d2bf8a463b58a194"
server_key_version: "encryptionkey@0"
I20260806 19:45:44.314579 4797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:44.335665 4797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:44.340435 4797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:44.340739 4797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:44.341995 4797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:44.343504 4797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:44.343600 4797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:44.343762 4797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:44.343851 4797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:44.387316 4797 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:46781
I20260806 19:45:44.387360 4929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:46781 every 8 connection(s)
I20260806 19:45:44.388998 4797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:45:44.398038 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 4797
I20260806 19:45:44.398695 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:36971
--local_ip_for_outbound_sockets=127.21.202.67
--tserver_master_addrs=127.21.202.126:34123
--webserver_port=39399
--webserver_interface=127.21.202.67
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:45:44 dist-test-slave-77v9 krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045544, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:44.421213 4932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:44.392274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:46111 (local address 127.21.202.126:34123)
0806 19:45:44.392893 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:44.392904 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:44.394109 (+ 1205us) server_negotiation.cc:407] Connection header received
0806 19:45:44.396371 (+ 2262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:44.396376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:44.396472 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:44.396577 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:44.398692 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:44.399589 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:44.401511 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:44.402278 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:44.402462 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:44.412648 (+ 10186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:44.412661 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:44.412670 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:44.412734 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:44.416871 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:44.418030 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:44.418039 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:44.418045 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:44.418228 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:44.418821 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:44.418829 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:44.418835 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:44.419267 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:44.419445 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:44.420382 (+ 937us) server_negotiation.cc:299] Negotiation successful
0806 19:45:44.420654 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":272,"threads_started":1}
I20260806 19:45:44.422027 4933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:44.393015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:46111)
0806 19:45:44.393862 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:44.393917 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:44.396025 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:44.396967 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:44.396977 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:44.397657 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:44.398467 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:44.398490 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:44.399798 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:44.399802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:44.401329 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:44.401346 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:44.402602 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:44.402613 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:44.402853 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:44.409130 (+ 6277us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:44.409172 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:44.412147 (+ 2975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:44.417144 (+ 4997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:44.417162 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:44.417179 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:44.417832 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:44.418397 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:44.418402 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:44.418406 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:44.418646 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:44.419434 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:44.419448 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:44.419936 (+ 488us) client_negotiation.cc:769] Sending connection context
0806 19:45:44.420210 (+ 274us) client_negotiation.cc:240] Negotiation successful
0806 19:45:44.420679 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":368,"threads_started":1}
I20260806 19:45:44.423777 4930 heartbeater.cc:347] Connected to a master server at 127.21.202.126:34123
I20260806 19:45:44.424144 4930 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:44.425031 4930 heartbeater.cc:511] Master 127.21.202.126:34123 requested a full tablet report, sending...
I20260806 19:45:44.427155 4605 ts_manager.cc:194] Registered new tserver with Master: beef8fd5b8d742489d9465e5ec6d8189 (127.21.202.66:46781)
I20260806 19:45:44.428361 4605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:46111
W20260806 19:45:44.803387 4934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:44.803810 4934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:44.804011 4934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:44.812985 4934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:44.813191 4934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:44.813385 4934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:45:44.826519 4934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:36971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=39399
--tserver_master_addrs=127.21.202.126:34123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:44.828563 4934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:44.830186 4934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:44.848542 4939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:44.848559 4940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:44.848601 4942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:44.849654 4934 server_base.cc:1034] running on GCE node
I20260806 19:45:44.850550 4934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:44.851785 4934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:44.853111 4934 hybrid_clock.cc:648] HybridClock initialized: now 1786045544853057 us; error 65 us; skew 500 ppm
I20260806 19:45:44.853628 4934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:44 dist-test-slave-77v9 krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045544, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:44.859061 4934 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:45:44.861094 4934 webserver.cc:460] Webserver started at http://127.21.202.67:39399/ using document root <none> and password file <none>
I20260806 19:45:44.861943 4934 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:44.862089 4934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:44.868609 4934 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 19:45:44.872056 4949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:44.874442 4934 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:45:44.874627 4934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "26160dc9a0fb4c2db9111aad5e9a02bb"
format_stamp: "Formatted at 2026-08-06 19:45:41 on dist-test-slave-77v9"
server_key: "4bb607f8f35ec1c3e7949ddd4ca3f09d"
server_key_iv: "da1b7f452a514f18c09b1ca01afbf5af"
server_key_version: "encryptionkey@0"
I20260806 19:45:44.876013 4934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:44.886384 4793 heartbeater.cc:503] Master 127.21.202.126:34123 was elected leader, sending a full tablet report...
I20260806 19:45:44.894197 4934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:44.899698 4934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:44.900344 4934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:44.902444 4934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:44.904215 4934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:44.904347 4934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:44.904443 4934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:44.904510 4934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:44.951515 4934 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:36971
I20260806 19:45:44.951562 5064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:36971 every 8 connection(s)
I20260806 19:45:44.953212 4934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:45:44.958300 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 4934
Aug 06 19:45:44 dist-test-slave-77v9 krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045544, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:45:44.983153 5067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:44.957361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:55857 (local address 127.21.202.126:34123)
0806 19:45:44.958126 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:44.958135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:44.961788 (+ 3653us) server_negotiation.cc:407] Connection header received
0806 19:45:44.961859 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:44.961863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:44.961967 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:44.962110 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:44.964889 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:44.966051 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:44.968534 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:44.969637 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:44.969809 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:44.974454 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:44.974460 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:44.974463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:44.974545 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:44.978512 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:44.980105 (+ 1593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:44.980120 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:44.980123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:44.980334 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:44.981040 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:44.981046 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:44.981049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:44.981455 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:44.981658 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:44.982456 (+ 798us) server_negotiation.cc:299] Negotiation successful
0806 19:45:44.982736 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":450,"threads_started":1}
I20260806 19:45:44.984277 5068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:44.958140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:55857)
0806 19:45:44.959065 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:44.959116 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:44.961415 (+ 2299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:44.962527 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:44.962568 (+ 41us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:44.963866 (+ 1298us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:45:44.964602 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:44.964624 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:44.966286 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:44.966296 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:44.968261 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:44.968286 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:44.969959 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:44.969966 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:44.970226 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:44.971121 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:45:44.971147 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:45:44.974132 (+ 2985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:45:44.978848 (+ 4716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:44.978870 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:44.978891 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:44.979890 (+ 999us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:44.980541 (+ 651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:45:44.980549 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:45:44.980556 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:45:44.980903 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:45:44.981635 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:45:44.981644 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:45:44.982056 (+ 412us) client_negotiation.cc:769] Sending connection context
0806 19:45:44.982349 (+ 293us) client_negotiation.cc:240] Negotiation successful
0806 19:45:44.982954 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":395,"threads_started":1}
I20260806 19:45:44.987277 5065 heartbeater.cc:347] Connected to a master server at 127.21.202.126:34123
I20260806 19:45:44.987687 5065 heartbeater.cc:464] Registering TS with master...
I20260806 19:45:44.988126 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:44.959323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46544 (local address 127.21.202.126:34123)
0806 19:45:44.966814 (+ 7491us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:44.966820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:44.967655 (+ 835us) server_negotiation.cc:407] Connection header received
0806 19:45:44.967859 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:44.967865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:44.967990 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:44.968156 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:44.970898 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:44.971726 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:44.974151 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:44.974897 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:44.975560 (+ 663us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:44.979630 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:44.979636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:44.979640 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:44.979682 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:44.983979 (+ 4297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:44.985085 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:44.985093 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:44.985098 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:44.985431 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:44.986134 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:44.986140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:44.986143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:44.986768 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:44.987105 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:44.987490 (+ 385us) server_negotiation.cc:299] Negotiation successful
0806 19:45:44.987741 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7314,"thread_start_us":7171,"threads_started":1}
I20260806 19:45:44.989022 5065 heartbeater.cc:511] Master 127.21.202.126:34123 requested a full tablet report, sending...
I20260806 19:45:44.991266 4604 ts_manager.cc:194] Registered new tserver with Master: 26160dc9a0fb4c2db9111aad5e9a02bb (127.21.202.67:36971)
I20260806 19:45:44.992512 4604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:55857
I20260806 19:45:45.006857 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:45:45.016705 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.008362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46546 (local address 127.21.202.126:34123)
0806 19:45:45.008951 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:45.008957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:45.009234 (+ 277us) server_negotiation.cc:407] Connection header received
0806 19:45:45.009551 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:45.009555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:45.009707 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:45.009907 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:45:45.010435 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.011223 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.012899 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.013357 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.013510 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.014804 (+ 1294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:45:45.015709 (+ 905us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:45:45.015953 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:45.016163 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 19:45:45.016300 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260806 19:45:45.019038 4604 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:46546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:45:45.027318 4604 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:45:45.065567 5076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.047437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:46781 (local address 127.0.0.1:40728)
0806 19:45:45.049256 (+ 1819us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:45.049315 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:45.049570 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:45.053867 (+ 4297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:45.053875 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:45.053907 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:45.054528 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:45.054546 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.057921 (+ 3375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.057932 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:45.061241 (+ 3309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:45.061271 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.064117 (+ 2846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.064126 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:45.064435 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.064540 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 19:45:45.064696 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 19:45:45.065018 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1479,"mutex_wait_us":3,"thread_start_us":3447,"threads_started":1}
I20260806 19:45:45.066526 5080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.049039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40728 (local address 127.21.202.66:46781)
0806 19:45:45.051743 (+ 2704us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:45.051759 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:45.051803 (+ 44us) server_negotiation.cc:407] Connection header received
0806 19:45:45.051943 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:45.051956 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:45.052334 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:45.054825 (+ 2491us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:45.055152 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.057596 (+ 2444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.061640 (+ 4044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.063753 (+ 2113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.065684 (+ 1931us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.065904 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:45.066001 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 19:45:45.066198 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2498,"thread_start_us":2379,"threads_started":1}
I20260806 19:45:45.070749 4863 tablet_service.cc:1467] Processing CreateTablet for tablet 6206f743c38c4fd1b8d92a8b4e5cdbb6 (DEFAULT_TABLE table=test-table [id=8a1a8ab8b680462f9a0b56d51029ba7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:45:45.072566 4863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6206f743c38c4fd1b8d92a8b4e5cdbb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:45.072655 5081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.049686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36658 (local address 127.21.202.67:36971)
0806 19:45:45.061992 (+ 12306us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:45.062007 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:45.063322 (+ 1315us) server_negotiation.cc:407] Connection header received
0806 19:45:45.063433 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:45.063441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:45.063666 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:45.063901 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:45.064893 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.066551 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.069341 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.070827 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.070995 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.071230 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:45.072070 (+ 840us) server_negotiation.cc:299] Negotiation successful
0806 19:45:45.072263 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11996,"thread_start_us":12345,"threads_started":1}
I20260806 19:45:45.072700 5079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.048146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:43791 (local address 127.0.0.1:52150)
0806 19:45:45.053403 (+ 5257us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:45.053435 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:45.053611 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:45.064465 (+ 10854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:45.064472 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:45.064500 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:45.065033 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:45.065046 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.067894 (+ 2848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.067900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:45.069931 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:45.069944 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.071848 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.071854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:45.072046 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.072085 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 19:45:45.072153 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 19:45:45.072289 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5001,"thread_start_us":14365,"threads_started":1}
I20260806 19:45:45.072422 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.048774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:36971 (local address 127.0.0.1:36658)
0806 19:45:45.062985 (+ 14211us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:45.063014 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:45.063185 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:45.063890 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:45.063895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:45.063923 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:45.064643 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:45.064656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.066742 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.066747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:45.069104 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:45.069128 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.071415 (+ 2287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.071422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:45.071604 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.071641 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:45:45.071743 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 19:45:45.071917 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13994,"mutex_wait_us":93,"thread_start_us":4031,"threads_started":1}
I20260806 19:45:45.073457 5078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.048443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52150 (local address 127.21.202.65:43791)
0806 19:45:45.063426 (+ 14983us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:45.063436 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:45.063471 (+ 35us) server_negotiation.cc:407] Connection header received
0806 19:45:45.063581 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:45.063589 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:45.063871 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:45.065311 (+ 1440us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:45.065606 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.067507 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.070133 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.071559 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.072520 (+ 961us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.072770 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:45.072904 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 19:45:45.073083 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14757,"thread_start_us":14459,"threads_started":1}
I20260806 19:45:45.076579 4998 tablet_service.cc:1467] Processing CreateTablet for tablet 6206f743c38c4fd1b8d92a8b4e5cdbb6 (DEFAULT_TABLE table=test-table [id=8a1a8ab8b680462f9a0b56d51029ba7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:45:45.077049 4728 tablet_service.cc:1467] Processing CreateTablet for tablet 6206f743c38c4fd1b8d92a8b4e5cdbb6 (DEFAULT_TABLE table=test-table [id=8a1a8ab8b680462f9a0b56d51029ba7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:45:45.078197 4998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6206f743c38c4fd1b8d92a8b4e5cdbb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:45.078450 4728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6206f743c38c4fd1b8d92a8b4e5cdbb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:45:45.090793 5082 tablet_bootstrap.cc:492] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189: Bootstrap starting.
I20260806 19:45:45.093369 5083 tablet_bootstrap.cc:492] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb: Bootstrap starting.
I20260806 19:45:45.096748 5082 tablet_bootstrap.cc:654] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:45.098263 5084 tablet_bootstrap.cc:492] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc: Bootstrap starting.
I20260806 19:45:45.098654 5083 tablet_bootstrap.cc:654] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:45.099901 5082 log.cc:824] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:45.100258 5083 log.cc:824] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:45.102409 5084 tablet_bootstrap.cc:654] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc: Neither blocks nor log segments found. Creating new log.
I20260806 19:45:45.103765 5082 tablet_bootstrap.cc:492] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189: No bootstrap required, opened a new log
I20260806 19:45:45.104117 5082 ts_tablet_manager.cc:1397] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260806 19:45:45.104605 5084 log.cc:824] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:45.104854 5083 tablet_bootstrap.cc:492] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb: No bootstrap required, opened a new log
I20260806 19:45:45.105384 5083 ts_tablet_manager.cc:1397] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260806 19:45:45.110145 5084 tablet_bootstrap.cc:492] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc: No bootstrap required, opened a new log
I20260806 19:45:45.110538 5084 ts_tablet_manager.cc:1397] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260806 19:45:45.116046 5082 raft_consensus.cc:348] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } } peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } }
I20260806 19:45:45.116592 5082 raft_consensus.cc:374] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:45.116768 5082 raft_consensus.cc:729] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beef8fd5b8d742489d9465e5ec6d8189, State: Initialized, Role: FOLLOWER
I20260806 19:45:45.118021 5082 consensus_queue.cc:260] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } } peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } }
I20260806 19:45:45.118767 5083 raft_consensus.cc:348] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } } peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } }
I20260806 19:45:45.119310 5083 raft_consensus.cc:374] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:45.119453 5083 raft_consensus.cc:729] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26160dc9a0fb4c2db9111aad5e9a02bb, State: Initialized, Role: FOLLOWER
I20260806 19:45:45.119756 4930 heartbeater.cc:503] Master 127.21.202.126:34123 was elected leader, sending a full tablet report...
I20260806 19:45:45.120319 5083 consensus_queue.cc:260] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } } peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } }
I20260806 19:45:45.120913 5082 ts_tablet_manager.cc:1428] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189: Time spent starting tablet: real 0.017s user 0.013s sys 0.003s
I20260806 19:45:45.122184 5084 raft_consensus.cc:348] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } } peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } }
I20260806 19:45:45.122550 5065 heartbeater.cc:503] Master 127.21.202.126:34123 was elected leader, sending a full tablet report...
I20260806 19:45:45.122730 5084 raft_consensus.cc:374] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:45:45.122851 5084 raft_consensus.cc:729] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6301b5c85d334595928148bee1ea2ddc, State: Initialized, Role: FOLLOWER
I20260806 19:45:45.123157 5083 ts_tablet_manager.cc:1428] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb: Time spent starting tablet: real 0.018s user 0.014s sys 0.002s
I20260806 19:45:45.123634 5084 consensus_queue.cc:260] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } } peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } }
I20260806 19:45:45.125833 5084 ts_tablet_manager.cc:1428] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
W20260806 19:45:45.141378 4931 tablet.cc:2369] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:45:45.178579 5090 raft_consensus.cc:482] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:45:45.179003 5090 raft_consensus.cc:504] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } } peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } }
I20260806 19:45:45.181003 5090 leader_election.cc:288] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers beef8fd5b8d742489d9465e5ec6d8189 (127.21.202.66:46781), 6301b5c85d334595928148bee1ea2ddc (127.21.202.65:43791)
I20260806 19:45:45.189196 5068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.181363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:46781 (local address 127.21.202.67:60965)
0806 19:45:45.181650 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:45.181672 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:45.181812 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:45.182354 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:45.182359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:45.182392 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:45.182625 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:45.182639 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.184667 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.184672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:45.186723 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:45.186739 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.188441 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.188445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:45.188582 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.188618 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 19:45:45.188715 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 19:45:45.188872 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260806 19:45:45.189333 5080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.181598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:60965 (local address 127.21.202.66:46781)
0806 19:45:45.181880 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:45.181892 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:45.181927 (+ 35us) server_negotiation.cc:407] Connection header received
0806 19:45:45.182070 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:45.182081 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:45.182179 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:45.182332 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:45.182812 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.184495 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.186965 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.188239 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.188450 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.188543 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:45.188795 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 19:45:45.189009 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 19:45:45.189918 5078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.181770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:51673 (local address 127.21.202.65:43791)
0806 19:45:45.181988 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:45.181997 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:45.182563 (+ 566us) server_negotiation.cc:407] Connection header received
0806 19:45:45.182663 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:45.182668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:45.182787 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:45.182923 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:45:45.184319 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.185317 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.187551 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.188935 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.189129 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.189210 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:45.189584 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 19:45:45.189701 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 19:45:45.190603 5092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.181533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:43791 (local address 127.21.202.67:51673)
0806 19:45:45.182413 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:45.182430 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:45:45.182557 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:45:45.183083 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:45:45.183089 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:45:45.183167 (+ 78us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:45:45.183774 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:45.183791 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.185485 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.185492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:45.187336 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:45:45.187347 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.189170 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.189179 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:45:45.189387 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.189419 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 19:45:45.190125 (+ 706us) client_negotiation.cc:240] Negotiation successful
0806 19:45:45.190273 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":476,"threads_started":1}
I20260806 19:45:45.190498 4883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6206f743c38c4fd1b8d92a8b4e5cdbb6" candidate_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beef8fd5b8d742489d9465e5ec6d8189" is_pre_election: true
I20260806 19:45:45.191073 4883 raft_consensus.cc:2454] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26160dc9a0fb4c2db9111aad5e9a02bb in term 0.
I20260806 19:45:45.191880 4950 leader_election.cc:302] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26160dc9a0fb4c2db9111aad5e9a02bb, beef8fd5b8d742489d9465e5ec6d8189; no voters:
I20260806 19:45:45.191762 4748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6206f743c38c4fd1b8d92a8b4e5cdbb6" candidate_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6301b5c85d334595928148bee1ea2ddc" is_pre_election: true
I20260806 19:45:45.192363 5090 raft_consensus.cc:2781] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:45:45.192346 4748 raft_consensus.cc:2454] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26160dc9a0fb4c2db9111aad5e9a02bb in term 0.
I20260806 19:45:45.192543 5090 raft_consensus.cc:482] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:45:45.192653 5090 raft_consensus.cc:3037] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:45.195302 5090 raft_consensus.cc:504] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } } peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } }
I20260806 19:45:45.196131 5090 leader_election.cc:288] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [CANDIDATE]: Term 1 election: Requested vote from peers beef8fd5b8d742489d9465e5ec6d8189 (127.21.202.66:46781), 6301b5c85d334595928148bee1ea2ddc (127.21.202.65:43791)
I20260806 19:45:45.196568 4883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6206f743c38c4fd1b8d92a8b4e5cdbb6" candidate_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beef8fd5b8d742489d9465e5ec6d8189"
I20260806 19:45:45.196736 4748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6206f743c38c4fd1b8d92a8b4e5cdbb6" candidate_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6301b5c85d334595928148bee1ea2ddc"
I20260806 19:45:45.196936 4883 raft_consensus.cc:3037] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:45.197005 4748 raft_consensus.cc:3037] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:45:45.200431 4748 raft_consensus.cc:2454] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26160dc9a0fb4c2db9111aad5e9a02bb in term 1.
I20260806 19:45:45.200431 4883 raft_consensus.cc:2454] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26160dc9a0fb4c2db9111aad5e9a02bb in term 1.
I20260806 19:45:45.201023 4950 leader_election.cc:302] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26160dc9a0fb4c2db9111aad5e9a02bb, beef8fd5b8d742489d9465e5ec6d8189; no voters:
I20260806 19:45:45.201442 5090 raft_consensus.cc:2781] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:45:45.202092 5090 raft_consensus.cc:686] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 1 LEADER]: Becoming Leader. State: Replica: 26160dc9a0fb4c2db9111aad5e9a02bb, State: Running, Role: LEADER
I20260806 19:45:45.202657 5090 consensus_queue.cc:237] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } } peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } }
W20260806 19:45:45.206022 5066 tablet.cc:2369] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:45:45.208937 4604 catalog_manager.cc:5495] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 26160dc9a0fb4c2db9111aad5e9a02bb (127.21.202.67). New cstate: current_term: 1 leader_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26160dc9a0fb4c2db9111aad5e9a02bb" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36971 } health_report { overall_health: HEALTHY } } }
I20260806 19:45:45.257977 5081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.251364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.21.202.67:36971)
0806 19:45:45.251571 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:45.251576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:45.251717 (+ 141us) server_negotiation.cc:407] Connection header received
0806 19:45:45.251987 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:45.251993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:45.252085 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:45.252240 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:45:45.252656 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.253632 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.255459 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.255994 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.256138 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.256669 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:45:45.257331 (+ 662us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:45:45.257528 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:45.257616 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 19:45:45.257701 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 19:45:45.268234 4748 raft_consensus.cc:1261] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc [term 1 FOLLOWER]: Refusing update from remote peer 26160dc9a0fb4c2db9111aad5e9a02bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:45:45.268290 4883 raft_consensus.cc:1261] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189 [term 1 FOLLOWER]: Refusing update from remote peer 26160dc9a0fb4c2db9111aad5e9a02bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:45:45.269459 5093 consensus_queue.cc:1035] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6301b5c85d334595928148bee1ea2ddc" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 43791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:45:45.269840 5090 consensus_queue.cc:1035] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "beef8fd5b8d742489d9465e5ec6d8189" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:45:45.284597 5095 mvcc.cc:205] Tried to move back new op lower bound from 7315642553406869504 to 7315642553154584576. Current Snapshot: MvccSnapshot[applied={T|T < 7315642553406869504}]
I20260806 19:45:45.306138 4605 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:46546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:45:45.306475 4605 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:46546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:45:45.311370 4605 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Sending DeleteTablet for 3 replicas of tablet 6206f743c38c4fd1b8d92a8b4e5cdbb6
I20260806 19:45:45.313063 4998 tablet_service.cc:1514] Processing DeleteTablet for tablet 6206f743c38c4fd1b8d92a8b4e5cdbb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:45:45 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:36658
I20260806 19:45:45.313054 4728 tablet_service.cc:1514] Processing DeleteTablet for tablet 6206f743c38c4fd1b8d92a8b4e5cdbb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:45:45 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:52150
I20260806 19:45:45.313346 4863 tablet_service.cc:1514] Processing DeleteTablet for tablet 6206f743c38c4fd1b8d92a8b4e5cdbb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:45:45 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:40728
I20260806 19:45:45.320302 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 4643
I20260806 19:45:45.326864 5106 tablet_replica.cc:330] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb: stopping tablet replica
I20260806 19:45:45.327623 5106 raft_consensus.cc:2229] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:45:45.328745 5106 raft_consensus.cc:2258] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:45:45.329145 5107 tablet_replica.cc:330] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189: stopping tablet replica
I20260806 19:45:45.329687 5107 raft_consensus.cc:2229] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:45:45.331485 5106 ts_tablet_manager.cc:1905] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:45:45.332187 5105 tablet_replica.cc:330] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc: stopping tablet replica
I20260806 19:45:45.332304 5107 raft_consensus.cc:2258] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:45:45.333069 5105 raft_consensus.cc:2229] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:45:45.334276 5105 raft_consensus.cc:2258] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:45:45.335228 5107 ts_tablet_manager.cc:1905] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:45:45.337697 5105 ts_tablet_manager.cc:1905] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:45:45.340968 5106 ts_tablet_manager.cc:1918] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:45:45.341230 5106 log.cc:1196] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/6206f743c38c4fd1b8d92a8b4e5cdbb6
I20260806 19:45:45.341938 5106 ts_tablet_manager.cc:1939] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 26160dc9a0fb4c2db9111aad5e9a02bb: Deleting consensus metadata
I20260806 19:45:45.342499 5078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.321722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52164 (local address 127.21.202.65:43791)
0806 19:45:45.321994 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:45.322004 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:45.322037 (+ 33us) server_negotiation.cc:407] Connection header received
0806 19:45:45.322144 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:45.322153 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:45.322289 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:45.322465 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:45.324921 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.326738 (+ 1817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.328898 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.329866 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.330048 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.333859 (+ 3811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:45.333876 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:45.333894 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:45.333950 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:45.338209 (+ 4259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:45.339440 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:45.339451 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:45.339454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:45.339673 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:45.340175 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:45.340180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:45.340183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:45.340789 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:45.341320 (+ 531us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:45.341645 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 19:45:45.341905 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 19:45:45.343473 4788 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:45.344024 5107 ts_tablet_manager.cc:1918] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:45:45.344259 5107 log.cc:1196] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/6206f743c38c4fd1b8d92a8b4e5cdbb6
I20260806 19:45:45.344336 4592 catalog_manager.cc:4841] TS 26160dc9a0fb4c2db9111aad5e9a02bb (127.21.202.67:36971): tablet 6206f743c38c4fd1b8d92a8b4e5cdbb6 (table test-table [id=8a1a8ab8b680462f9a0b56d51029ba7b]) successfully deleted
I20260806 19:45:45.344962 5107 ts_tablet_manager.cc:1939] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P beef8fd5b8d742489d9465e5ec6d8189: Deleting consensus metadata
I20260806 19:45:45.348269 4589 catalog_manager.cc:4841] TS beef8fd5b8d742489d9465e5ec6d8189 (127.21.202.66:46781): tablet 6206f743c38c4fd1b8d92a8b4e5cdbb6 (table test-table [id=8a1a8ab8b680462f9a0b56d51029ba7b]) successfully deleted
I20260806 19:45:45.465687 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 4643
I20260806 19:45:45.493234 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 4797
W20260806 19:45:45.493532 4590 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.202.65:43791 (error 108)
I20260806 19:45:45.495731 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.495156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:43791 (local address 127.0.0.1:52168)
0806 19:45:45.495421 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:45.495547 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:43791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260806 19:45:45.495994 4590 catalog_manager.cc:4584] TS 6301b5c85d334595928148bee1ea2ddc (127.21.202.65:43791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6206f743c38c4fd1b8d92a8b4e5cdbb6: Network error: Client connection negotiation failed: client connection to 127.21.202.65:43791: connect: Connection refused (error 111)
I20260806 19:45:45.512138 5080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.494210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40738 (local address 127.21.202.66:46781)
0806 19:45:45.494821 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:45.494829 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:45.494853 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:45:45.494939 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:45.494945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:45.495046 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:45.495169 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:45.496859 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.497662 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.499107 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.499532 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.499684 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.503537 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:45.503558 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:45.503576 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:45.503624 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:45.508203 (+ 4579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:45.509381 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:45.509399 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:45.509402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:45.509559 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:45.509982 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:45.509987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:45.509991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:45.510435 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:45.510760 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:45.511210 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 19:45:45.511487 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480}
I20260806 19:45:45.513644 4925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:45.550060 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.549420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:43791 (local address 127.0.0.1:52180)
0806 19:45:45.549699 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:45.549840 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:43791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 19:45:45.551365 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.550913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:43791 (local address 127.0.0.1:52186)
0806 19:45:45.551097 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:45.551208 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:43791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 19:45:45.612272 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.611619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:43791 (local address 127.0.0.1:52194)
0806 19:45:45.611924 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:45.612049 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:43791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260806 19:45:45.613690 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.613200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:43791 (local address 127.0.0.1:52206)
0806 19:45:45.613384 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:45.613487 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:43791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 19:45:45.627396 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 4797
I20260806 19:45:45.653801 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 4934
I20260806 19:45:45.675940 5081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.654802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36676 (local address 127.21.202.67:36971)
0806 19:45:45.655131 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0806 19:45:45.655139 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:45:45.655192 (+ 53us) server_negotiation.cc:407] Connection header received
0806 19:45:45.655336 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:45:45.655342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:45:45.655443 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:45:45.655571 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:45:45.657717 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.659167 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.661343 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:45:45.662228 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:45:45.662457 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:45:45.666582 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:45:45.666611 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:45:45.666635 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:45:45.666721 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:45:45.671651 (+ 4930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:45.673149 (+ 1498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:45.673173 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:45.673177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:45.673412 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:45:45.673946 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:45:45.673951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:45:45.673955 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:45:45.674489 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:45:45.674864 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0806 19:45:45.675216 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 19:45:45.675498 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194}
I20260806 19:45:45.676877 5059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:45.695026 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.694257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:43791 (local address 127.0.0.1:52222)
0806 19:45:45.694678 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:45.694799 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:43791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231}
I20260806 19:45:45.696343 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.695869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:43791 (local address 127.0.0.1:52228)
0806 19:45:45.696129 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:45.696210 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:43791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260806 19:45:45.829939 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 4934
I20260806 19:45:45.857156 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 4572
I20260806 19:45:45.858618 4636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:45:45.982223 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:45.981526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:43791 (local address 127.0.0.1:52238)
0806 19:45:45.981820 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:45.981988 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:43791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260806 19:45:45.982764 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 4572
I20260806 19:45:46.009685 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:34123
--webserver_interface=127.21.202.126
--webserver_port=44365
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:34123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:45:46.432159 5113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:46.432503 5113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:46.432598 5113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:46.440531 5113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:45:46.440698 5113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:46.440761 5113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:46.440865 5113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:45:46.440935 5113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:45:46.453107 5113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:34123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:34123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=44365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:46.454681 5113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:46.456135 5113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:46.473201 5118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:46.473630 5119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:46.473908 5121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:46.475504 5113 server_base.cc:1034] running on GCE node
I20260806 19:45:46.476580 5113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:46.478492 5113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:46.479974 5113 hybrid_clock.cc:648] HybridClock initialized: now 1786045546479876 us; error 67 us; skew 500 ppm
I20260806 19:45:46.481208 5113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:46 dist-test-slave-77v9 krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045546, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:46.486263 5113 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:45:46.489120 5113 webserver.cc:460] Webserver started at http://127.21.202.126:44365/ using document root <none> and password file <none>
I20260806 19:45:46.490032 5113 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:46.490208 5113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:46.499030 5113 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 19:45:46.504355 5128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:46.506651 5113 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 19:45:46.506945 5113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "e776789a4e954d2f9118c36dee9e169a"
format_stamp: "Formatted at 2026-08-06 19:45:39 on dist-test-slave-77v9"
server_key: "172ac4ee59ab68bb23f93a874df130d6"
server_key_iv: "b09e59413308bed0a0f7d60855f3529d"
server_key_version: "encryptionkey@0"
I20260806 19:45:46.508378 5113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:46.539105 5113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:46.546196 5113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:46.546581 5113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:46.573112 5113 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:34123
I20260806 19:45:46.573333 5180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:34123 every 8 connection(s)
I20260806 19:45:46.574849 5113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:45:46.581995 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 5113
I20260806 19:45:46.582985 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:43791
--local_ip_for_outbound_sockets=127.21.202.65
--tserver_master_addrs=127.21.202.126:34123
--webserver_port=37065
--webserver_interface=127.21.202.65
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:45:46.584493 5182 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:45:46.588277 5182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Bootstrap starting.
I20260806 19:45:46.613422 5182 log.cc:824] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:45:46.635179 5182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:45:46.635869 5182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a: Bootstrap complete.
I20260806 19:45:46.646618 5182 raft_consensus.cc:348] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } }
I20260806 19:45:46.647985 5182 raft_consensus.cc:729] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e776789a4e954d2f9118c36dee9e169a, State: Initialized, Role: FOLLOWER
I20260806 19:45:46.649283 5182 consensus_queue.cc:260] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } }
I20260806 19:45:46.649781 5182 raft_consensus.cc:388] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:45:46.649958 5182 raft_consensus.cc:482] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:45:46.650162 5182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 2 FOLLOWER]: Advancing to term 3
I20260806 19:45:46.656778 5182 raft_consensus.cc:504] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } }
I20260806 19:45:46.657583 5182 leader_election.cc:302] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e776789a4e954d2f9118c36dee9e169a; no voters:
I20260806 19:45:46.658401 5182 leader_election.cc:288] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 19:45:46.658576 5186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 3 FOLLOWER]: Leader election won for term 3
I20260806 19:45:46.660336 5186 raft_consensus.cc:686] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [term 3 LEADER]: Becoming Leader. State: Replica: e776789a4e954d2f9118c36dee9e169a, State: Running, Role: LEADER
I20260806 19:45:46.661948 5182 sys_catalog.cc:564] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:45:46.661733 5186 consensus_queue.cc:237] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } }
I20260806 19:45:46.665930 5188 sys_catalog.cc:455] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e776789a4e954d2f9118c36dee9e169a. Latest consensus state: current_term: 3 leader_uuid: "e776789a4e954d2f9118c36dee9e169a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } } }
I20260806 19:45:46.667776 5187 sys_catalog.cc:455] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e776789a4e954d2f9118c36dee9e169a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776789a4e954d2f9118c36dee9e169a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 34123 } } }
I20260806 19:45:46.668336 5187 sys_catalog.cc:458] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:46.668510 5188 sys_catalog.cc:458] T 00000000000000000000000000000000 P e776789a4e954d2f9118c36dee9e169a [sys.catalog]: This master's current role is: LEADER
I20260806 19:45:46.681406 5194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:45:46.693974 5194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:45:46.698506 5194 catalog_manager.cc:1259] Loaded cluster ID: 555867a9580c4a968916979338e7b614
I20260806 19:45:46.698733 5194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 19:45:46.702376 5194 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4c721209ad google::LogMessage::Fail() at ??:0
@ 0x7f4c7212388a google::LogMessage::SendToLog() at ??:0
@ 0x7f4c721204de google::LogMessage::Flush() at ??:0
@ 0x7f4c72121a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4c87b53512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f4c87b50976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4c87bd8610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f4c87bd8439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4c74d6c345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4c74d6f91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f4c74d6f799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4c74d4aeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4c780246db start_thread at ??:0
@ 0x7f4c70fba61f clone at ??:0
W20260806 19:45:47.116894 5184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:47.117453 5184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:47.117712 5184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:47.127163 5184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:47.127399 5184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:47.127614 5184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:45:47.140777 5184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:43791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=37065
--tserver_master_addrs=127.21.202.126:34123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:47.143678 5184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:47.146341 5184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:47.165570 5223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:47.165818 5224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:47.166369 5226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:47.167182 5184 server_base.cc:1034] running on GCE node
I20260806 19:45:47.168467 5184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:47.169906 5184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:47.171356 5184 hybrid_clock.cc:648] HybridClock initialized: now 1786045547171277 us; error 83 us; skew 500 ppm
I20260806 19:45:47.171811 5184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:47 dist-test-slave-77v9 krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:47.176206 5184 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:45:47.178097 5184 webserver.cc:460] Webserver started at http://127.21.202.65:37065/ using document root <none> and password file <none>
I20260806 19:45:47.178912 5184 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:47.179088 5184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:47.186946 5184 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260806 19:45:47.192447 5233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:47.194883 5184 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260806 19:45:47.195173 5184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "6301b5c85d334595928148bee1ea2ddc"
format_stamp: "Formatted at 2026-08-06 19:45:40 on dist-test-slave-77v9"
server_key: "7054168269b467c145429271684e735b"
server_key_iv: "ddc65fa2849dac686fe56bb2fdbe79e1"
server_key_version: "encryptionkey@0"
I20260806 19:45:47.196460 5184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:47.221700 5184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:47.228539 5184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:47.228927 5184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:47.230183 5184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:47.232753 5240 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260806 19:45:47.239352 5240 ts_tablet_manager.cc:1852] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260806 19:45:47.240869 5240 ts_tablet_manager.cc:1905] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:45:47.251299 5240 ts_tablet_manager.cc:1918] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:45:47.251609 5240 log.cc:1196] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/6206f743c38c4fd1b8d92a8b4e5cdbb6
I20260806 19:45:47.252243 5240 ts_tablet_manager.cc:1939] T 6206f743c38c4fd1b8d92a8b4e5cdbb6 P 6301b5c85d334595928148bee1ea2ddc: Deleting consensus metadata
I20260806 19:45:47.252746 5184 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260806 19:45:47.253137 5184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.021s user 0.001s sys 0.000s
I20260806 19:45:47.253316 5184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:47.253423 5184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:47.300222 5184 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:43791
I20260806 19:45:47.300292 5347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:43791 every 8 connection(s)
I20260806 19:45:47.301767 5184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:45:47.308619 5350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:47.305328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:53061)
0806 19:45:47.306824 (+ 1496us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:47.307263 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1000,"thread_start_us":578,"threads_started":1}
W20260806 19:45:47.309648 5348 heartbeater.cc:644] Failed to heartbeat to 127.21.202.126:34123 (0 consecutive failures): Network error: Failed to ping master at 127.21.202.126:34123: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
I20260806 19:45:47.310210 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 5184
I20260806 19:45:47.311115 5350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:47.310407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:48321)
0806 19:45:47.310686 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:47.310816 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 19:45:47.311170 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:46781
--local_ip_for_outbound_sockets=127.21.202.66
--tserver_master_addrs=127.21.202.126:34123
--webserver_port=38499
--webserver_interface=127.21.202.66
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:45:47.312500 5350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:47.311938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:33223)
0806 19:45:47.312166 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:47.312291 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 19:45:47.312863 5348 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 19:45:47.742432 5351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:47.742774 5351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:47.742972 5351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:47.751552 5351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:47.751729 5351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:47.751951 5351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:45:47.768076 5351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:46781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=38499
--tserver_master_addrs=127.21.202.126:34123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:47.770833 5351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:47.773000 5351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:47.794509 5356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:47.794799 5358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:47.795467 5360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:47.798687 5351 server_base.cc:1034] running on GCE node
I20260806 19:45:47.800128 5351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:47.802464 5351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:47.804116 5351 hybrid_clock.cc:648] HybridClock initialized: now 1786045547804010 us; error 96 us; skew 500 ppm
I20260806 19:45:47.804682 5351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:47 dist-test-slave-77v9 krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:47.810724 5351 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:45:47.813390 5351 webserver.cc:460] Webserver started at http://127.21.202.66:38499/ using document root <none> and password file <none>
I20260806 19:45:47.814615 5351 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:47.814833 5351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:47.824288 5351 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:45:47.828542 5367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:47.830397 5351 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 19:45:47.830574 5351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "beef8fd5b8d742489d9465e5ec6d8189"
format_stamp: "Formatted at 2026-08-06 19:45:40 on dist-test-slave-77v9"
server_key: "e4ca53ebed0bb6a09403a82b104f6858"
server_key_iv: "6887b4cf41263235d2bf8a463b58a194"
server_key_version: "encryptionkey@0"
I20260806 19:45:47.831617 5351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:47.864006 5351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:47.870898 5351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:47.871212 5351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:47.872524 5351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:47.874430 5351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:47.874550 5351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:47.874744 5351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:47.874872 5351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:47.922262 5351 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:46781
I20260806 19:45:47.922328 5480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:46781 every 8 connection(s)
I20260806 19:45:47.924734 5351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:45:47.930233 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 5351
I20260806 19:45:47.930985 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:36971
--local_ip_for_outbound_sockets=127.21.202.67
--tserver_master_addrs=127.21.202.126:34123
--webserver_port=39399
--webserver_interface=127.21.202.67
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:45:47.932240 5483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:47.928629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:49489)
0806 19:45:47.930174 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:47.930640 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1104,"thread_start_us":658,"threads_started":1}
W20260806 19:45:47.933856 5481 heartbeater.cc:644] Failed to heartbeat to 127.21.202.126:34123 (0 consecutive failures): Network error: Failed to ping master at 127.21.202.126:34123: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
I20260806 19:45:47.935258 5483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:47.934777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:56981)
0806 19:45:47.935007 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:47.935101 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 19:45:47.936558 5483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:47.936059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:51771)
0806 19:45:47.936231 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:47.936325 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 19:45:47.936937 5481 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 19:45:48.314678 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:48.313596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:42895)
0806 19:45:48.314274 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:48.314384 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":373,"threads_started":1}
W20260806 19:45:48.334513 5484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:45:48.334930 5484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:45:48.335127 5484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:45:48.343778 5484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:45:48.343983 5484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:45:48.344332 5484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:45:48.355540 5484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:36971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=39399
--tserver_master_addrs=127.21.202.126:34123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:45:48.357331 5484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:45:48.358847 5484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:45:48.374908 5492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:45:48.375005 5491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:48.375454 5484 server_base.cc:1034] running on GCE node
W20260806 19:45:48.375527 5494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:45:48.376292 5484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:45:48.377451 5484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:45:48.378759 5484 hybrid_clock.cc:648] HybridClock initialized: now 1786045548378677 us; error 61 us; skew 500 ppm
I20260806 19:45:48.379179 5484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:45:48 dist-test-slave-77v9 krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045548, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:45:48.382777 5484 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:45:48.384567 5484 webserver.cc:460] Webserver started at http://127.21.202.67:39399/ using document root <none> and password file <none>
I20260806 19:45:48.385762 5484 fs_manager.cc:359] Metadata directory not provided
I20260806 19:45:48.385993 5484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:45:48.392225 5484 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 19:45:48.396292 5501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:48.398526 5484 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.003s
I20260806 19:45:48.398789 5484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "26160dc9a0fb4c2db9111aad5e9a02bb"
format_stamp: "Formatted at 2026-08-06 19:45:41 on dist-test-slave-77v9"
server_key: "4bb607f8f35ec1c3e7949ddd4ca3f09d"
server_key_iv: "da1b7f452a514f18c09b1ca01afbf5af"
server_key_version: "encryptionkey@0"
I20260806 19:45:48.399901 5484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:45:48.415182 5484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:45:48.420248 5484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:45:48.420598 5484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:45:48.421854 5484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:45:48.423713 5484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:45:48.423833 5484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:48.424000 5484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:45:48.424088 5484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:45:48.469885 5484 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:36971
I20260806 19:45:48.469986 5614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:36971 every 8 connection(s)
I20260806 19:45:48.472613 5484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:45:48.480587 5617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:48.476763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:39945)
0806 19:45:48.478517 (+ 1754us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:48.479112 (+ 595us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":859,"thread_start_us":397,"threads_started":1}
I20260806 19:45:48.482170 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 5484
W20260806 19:45:48.482286 5615 heartbeater.cc:644] Failed to heartbeat to 127.21.202.126:34123 (0 consecutive failures): Network error: Failed to ping master at 127.21.202.126:34123: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
I20260806 19:45:48.483994 5617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:48.483308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:35629)
0806 19:45:48.483626 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:48.483781 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 19:45:48.485903 5617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:48.485281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:35625)
0806 19:45:48.485555 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:48.485689 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260806 19:45:48.486384 5615 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 19:45:48.938525 5619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:48.937550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:55287)
0806 19:45:48.938123 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:48.938251 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":269,"threads_started":1}
I20260806 19:45:49.316910 5620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:49.315931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:52691)
0806 19:45:49.316548 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:49.316642 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":376,"threads_started":1}
I20260806 19:45:49.489077 5621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:49.487493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:44635)
0806 19:45:49.488488 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:49.488701 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":846,"thread_start_us":705,"threads_started":1}
I20260806 19:45:49.940500 5622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:49.939635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:43175)
0806 19:45:49.940188 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:49.940307 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":280,"threads_started":1}
I20260806 19:45:50.319725 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:50.318351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:45543)
0806 19:45:50.319368 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:50.319486 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":863,"thread_start_us":532,"threads_started":1}
I20260806 19:45:50.491515 5624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:50.490228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:41793)
0806 19:45:50.491159 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:50.491286 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":384,"threads_started":1}
I20260806 19:45:50.942677 5625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:50.941831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:36659)
0806 19:45:50.942371 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:50.942480 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":285,"threads_started":1}
I20260806 19:45:51.321719 5626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:51.320911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:56055)
0806 19:45:51.321448 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:51.321537 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":277,"threads_started":1}
I20260806 19:45:51.494240 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:51.493335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:38905)
0806 19:45:51.493961 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:51.494059 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":278,"threads_started":1}
I20260806 19:45:51.944599 5628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:51.943748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:48693)
0806 19:45:51.944296 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:51.944402 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":286,"threads_started":1}
I20260806 19:45:52.323729 5629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:52.322882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:36377)
0806 19:45:52.323448 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:52.323545 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":279,"threads_started":1}
I20260806 19:45:52.497083 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:52.495768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:43831)
0806 19:45:52.496473 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:52.496781 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":405,"threads_started":1}
I20260806 19:45:52.947221 5631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:52.946056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:34793)
0806 19:45:52.946873 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:52.946997 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":365,"threads_started":1}
I20260806 19:45:53.326254 5632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:53.325383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:50299)
0806 19:45:53.325971 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:53.326067 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":294,"threads_started":1}
I20260806 19:45:53.499320 5633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:53.498492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:57331)
0806 19:45:53.499037 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:53.499127 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":277,"threads_started":1}
I20260806 19:45:53.949510 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:53.948521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:60153)
0806 19:45:53.949227 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:53.949338 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":395,"threads_started":1}
I20260806 19:45:54.328791 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:54.327791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:51485)
0806 19:45:54.328453 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:54.328558 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":351,"threads_started":1}
I20260806 19:45:54.501447 5636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:54.500447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:56947)
0806 19:45:54.501117 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:54.501237 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":307,"threads_started":1}
I20260806 19:45:54.951728 5638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:54.950773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:54103)
0806 19:45:54.951382 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:54.951487 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":275,"threads_started":1}
I20260806 19:45:55.330940 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:55.330161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:36523)
0806 19:45:55.330669 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:55.330765 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":289,"threads_started":1}
I20260806 19:45:55.503875 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:55.502807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:49947)
0806 19:45:55.503554 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:55.503645 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":333,"threads_started":1}
I20260806 19:45:55.953862 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:55.952991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:49203)
0806 19:45:55.953562 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:55.953665 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":297,"threads_started":1}
I20260806 19:45:56.333148 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:56.332237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:56695)
0806 19:45:56.332835 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:56.332947 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":296,"threads_started":1}
I20260806 19:45:56.506361 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:56.505427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:59423)
0806 19:45:56.506092 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:56.506189 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":363,"threads_started":1}
I20260806 19:45:56.956434 5644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:56.955511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:33945)
0806 19:45:56.956127 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:56.956225 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":326,"threads_started":1}
I20260806 19:45:57.335757 5645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:57.334459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:55753)
0806 19:45:57.335201 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:57.335343 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":381,"threads_started":1}
I20260806 19:45:57.508416 5646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:57.507532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:32917)
0806 19:45:57.508125 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:57.508222 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":271,"threads_started":1}
I20260806 19:45:57.958968 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:57.958035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:42613)
0806 19:45:57.958642 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:57.958743 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":328,"threads_started":1}
I20260806 19:45:58.338099 5648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:58.337111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:59337)
0806 19:45:58.337739 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:58.337874 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":353,"threads_started":1}
I20260806 19:45:58.510646 5649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:58.509841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:44647)
0806 19:45:58.510379 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:58.510470 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":292,"threads_started":1}
I20260806 19:45:58.961190 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:58.960183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:55865)
0806 19:45:58.960767 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:58.960970 (+ 203us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":303,"threads_started":1}
I20260806 19:45:59.339946 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:59.339147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:37071)
0806 19:45:59.339628 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:59.339710 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":269,"threads_started":1}
I20260806 19:45:59.512907 5652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:59.511863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:36289)
0806 19:45:59.512505 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:59.512613 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":353,"threads_started":1}
I20260806 19:45:59.963498 5653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:45:59.962282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:34499)
0806 19:45:59.963111 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0806 19:45:59.963250 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":461,"threads_started":1}
I20260806 19:46:00.342644 5654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:00.341670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:52109)
0806 19:46:00.342306 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:00.342441 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":339,"threads_started":1}
I20260806 19:46:00.515115 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:00.514156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:41271)
0806 19:46:00.514791 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:00.514947 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":283,"threads_started":1}
I20260806 19:46:00.966150 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:00.965152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:50467)
0806 19:46:00.965759 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:00.965919 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":311,"threads_started":1}
I20260806 19:46:01.344951 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:01.344081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:58489)
0806 19:46:01.344598 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:01.344689 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":280,"threads_started":1}
I20260806 19:46:01.517652 5658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:01.516580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:44611)
0806 19:46:01.517274 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:01.517377 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":364,"threads_started":1}
I20260806 19:46:01.968796 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:01.967765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:50523)
0806 19:46:01.968359 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:01.968574 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":360,"threads_started":1}
I20260806 19:46:02.347206 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:02.346327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:38055)
0806 19:46:02.346882 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:02.346995 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":283,"threads_started":1}
I20260806 19:46:02.519963 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:02.519069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:47835)
0806 19:46:02.519637 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:02.519742 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":317,"threads_started":1}
I20260806 19:46:02.971801 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:02.970596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.66:38049)
0806 19:46:02.971459 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:02.971597 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":586,"threads_started":1}
I20260806 19:46:03.349812 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:03.348795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.65:58541)
0806 19:46:03.349562 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:03.349650 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":331,"threads_started":1}
I20260806 19:46:03.490015 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 5184
I20260806 19:46:03.522495 5665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:03.521604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:34123 (local address 127.21.202.67:53581)
0806 19:46:03.522180 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:03.522301 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:34123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":319,"threads_started":1}
I20260806 19:46:03.522773 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:03.491956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47966 (local address 127.21.202.65:43791)
0806 19:46:03.493013 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:03.493037 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:03.493094 (+ 57us) server_negotiation.cc:407] Connection header received
0806 19:46:03.496318 (+ 3224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:03.496339 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:03.497204 (+ 865us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:03.499015 (+ 1811us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:03.501488 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:03.503025 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:03.504532 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:03.505354 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:03.505614 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:03.514694 (+ 9080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:03.514705 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:03.514725 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:03.514779 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:03.518369 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:03.519333 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:03.519341 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:03.519348 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:03.519538 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:03.519945 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:03.519952 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:03.519956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:03.520559 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:03.521039 (+ 480us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:03.521672 (+ 633us) server_negotiation.cc:299] Negotiation successful
0806 19:46:03.522348 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":544,"threads_started":1}
I20260806 19:46:03.524472 5343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:46:03.620710 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 5184
I20260806 19:46:03.644006 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 5351
I20260806 19:46:03.672798 5667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:03.645774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32790 (local address 127.21.202.66:46781)
0806 19:46:03.646435 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:03.646449 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:03.646503 (+ 54us) server_negotiation.cc:407] Connection header received
0806 19:46:03.648192 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:03.648203 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:03.648717 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:03.649243 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:03.651402 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:03.653720 (+ 2318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:03.655755 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:03.657658 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:03.657874 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:03.661454 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:03.661463 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:03.661484 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:03.661543 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:03.666009 (+ 4466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:03.667264 (+ 1255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:03.667278 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:03.667291 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:03.667845 (+ 554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:03.668777 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:03.668784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:03.668787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:03.670443 (+ 1656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:03.670934 (+ 491us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:03.671775 (+ 841us) server_negotiation.cc:299] Negotiation successful
0806 19:46:03.672363 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":293,"threads_started":1}
I20260806 19:46:03.674914 5476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:46:03.779462 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 5351
I20260806 19:46:03.804214 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 5484
I20260806 19:46:03.826416 5669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:03.805281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51308 (local address 127.21.202.67:36971)
0806 19:46:03.805873 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:03.805888 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:03.805934 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:46:03.807772 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:03.807784 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:03.808271 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:03.808679 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:03.810961 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:03.812291 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:03.813767 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:03.814554 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:03.814780 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:03.817882 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:03.817891 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:03.817910 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:03.817964 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:03.822214 (+ 4250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:03.823187 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:03.823195 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:03.823203 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:03.823403 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:03.823853 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:03.823863 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:03.823865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:03.824528 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:03.824945 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:03.825412 (+ 467us) server_negotiation.cc:299] Negotiation successful
0806 19:46:03.826058 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":272,"threads_started":1}
I20260806 19:46:03.828406 5610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:46:03.920145 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 5484
2026-08-06T19:46:03Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (26609 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:46:04 dist-test-slave-77v9 krb5kdc[5673](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:46:04 dist-test-slave-77v9 krb5kdc[5673](info): set up 2 sockets
Aug 06 19:46:04 dist-test-slave-77v9 krb5kdc[5673](info): commencing operation
krb5kdc: starting...
W20260806 19:46:05.143400 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:46:05 dist-test-slave-77v9 krb5kdc[5673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:46:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:46:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:46:05.502744 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46849
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:46849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:46:05.893635 5686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:46:05.893986 5686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:46:05.894080 5686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:46:05.902838 5686 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 19:46:05.902972 5686 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 19:46:05.903012 5686 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 19:46:05.903136 5686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:46:05.903247 5686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:46:05.903345 5686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:46:05.903421 5686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:46:05.903515 5686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:46:05.915582 5686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:46849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:46:05.918053 5686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:46:05.920413 5686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:46:05.935904 5692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:05.935855 5694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:05.935966 5691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:46:05.936618 5686 server_base.cc:1034] running on GCE node
I20260806 19:46:05.937584 5686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:46:05.939745 5686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:46:05.941169 5686 hybrid_clock.cc:648] HybridClock initialized: now 1786045565941065 us; error 69 us; skew 500 ppm
I20260806 19:46:05.941643 5686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:46:05 dist-test-slave-77v9 krb5kdc[5673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:46:05.946110 5686 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:46:05.948058 5686 webserver.cc:460] Webserver started at http://127.21.202.126:44555/ using document root <none> and password file <none>
I20260806 19:46:05.948971 5686 fs_manager.cc:359] Metadata directory not provided
I20260806 19:46:05.949072 5686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:46:05.949361 5686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:46:05.952481 5686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "7d4dc71690334648b218d29a91b1335a"
format_stamp: "Formatted at 2026-08-06 19:46:05 on dist-test-slave-77v9"
server_key: "8c892914178612f5253e810bc6aa7086"
server_key_iv: "20462370340cc7784d3c98aab4fa70e8"
server_key_version: "encryptionkey@0"
I20260806 19:46:05.953383 5686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "7d4dc71690334648b218d29a91b1335a"
format_stamp: "Formatted at 2026-08-06 19:46:05 on dist-test-slave-77v9"
server_key: "8c892914178612f5253e810bc6aa7086"
server_key_iv: "20462370340cc7784d3c98aab4fa70e8"
server_key_version: "encryptionkey@0"
I20260806 19:46:05.959424 5686 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 19:46:05.963841 5701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:05.965664 5686 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:46:05.965960 5686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "7d4dc71690334648b218d29a91b1335a"
format_stamp: "Formatted at 2026-08-06 19:46:05 on dist-test-slave-77v9"
server_key: "8c892914178612f5253e810bc6aa7086"
server_key_iv: "20462370340cc7784d3c98aab4fa70e8"
server_key_version: "encryptionkey@0"
I20260806 19:46:05.966205 5686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:46:05.993275 5686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:46:05.999476 5686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:46:05.999892 5686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:46:06.024582 5686 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:46849
I20260806 19:46:06.024600 5752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:46849 every 8 connection(s)
I20260806 19:46:06.026142 5686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:46:06.030314 5753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:46:06.031085 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 5686
I20260806 19:46:06.031327 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:46:06.032220 22313 external_mini_cluster.cc:1428] Setting key a6a3033e3dac38df0f14ab21ec805aac
I20260806 19:46:06.048930 5753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Bootstrap starting.
Aug 06 19:46:06 dist-test-slave-77v9 krb5kdc[5673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:46:06.052865 5753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Neither blocks nor log segments found. Creating new log.
I20260806 19:46:06.054101 5753 log.cc:824] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:46:06.058701 5753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: No bootstrap required, opened a new log
I20260806 19:46:06.063033 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:06.034848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49344 (local address 127.21.202.126:46849)
0806 19:46:06.035766 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:06.035779 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:06.035832 (+ 53us) server_negotiation.cc:407] Connection header received
0806 19:46:06.037679 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:06.037706 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:06.038175 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:06.038561 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:06.040648 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:06.042129 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:06.044292 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:06.045180 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:06.045339 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:06.051797 (+ 6458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:06.051813 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:06.051837 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:06.051894 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:06.056657 (+ 4763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:06.057977 (+ 1320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:06.057988 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:06.057997 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:06.058298 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:06.059432 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:06.059439 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:06.059442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:06.060202 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:06.060624 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:06.061229 (+ 605us) server_negotiation.cc:299] Negotiation successful
0806 19:46:06.061765 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":434,"threads_started":1}
I20260806 19:46:06.071024 5753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } }
I20260806 19:46:06.071537 5753 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:46:06.071699 5753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d4dc71690334648b218d29a91b1335a, State: Initialized, Role: FOLLOWER
I20260806 19:46:06.072329 5753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } }
I20260806 19:46:06.072610 5753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:46:06.072763 5753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:46:06.073014 5753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:46:06.075610 5753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } }
I20260806 19:46:06.076179 5753 leader_election.cc:302] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d4dc71690334648b218d29a91b1335a; no voters:
I20260806 19:46:06.076956 5753 leader_election.cc:288] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:46:06.077214 5758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:46:06.078410 5758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 1 LEADER]: Becoming Leader. State: Replica: 7d4dc71690334648b218d29a91b1335a, State: Running, Role: LEADER
I20260806 19:46:06.079110 5758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } }
I20260806 19:46:06.079464 5753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:46:06.082446 5760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d4dc71690334648b218d29a91b1335a. Latest consensus state: current_term: 1 leader_uuid: "7d4dc71690334648b218d29a91b1335a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } } }
I20260806 19:46:06.082585 5759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d4dc71690334648b218d29a91b1335a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } } }
I20260806 19:46:06.083038 5760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: This master's current role is: LEADER
I20260806 19:46:06.083071 5759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: This master's current role is: LEADER
I20260806 19:46:06.085443 5765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:46:06.092800 5765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:46:06.104423 5765 catalog_manager.cc:1347] Generated new cluster ID: e038041afe914edc8f292fb2336fcf0a
I20260806 19:46:06.104600 5765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:46:06.116523 5765 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:46:06.118137 5765 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:46:06.131325 5765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Generated new TSK 0
I20260806 19:46:06.132315 5765 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:46:06.222674 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46849
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:46:06.628948 5780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:46:06.629283 5780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:46:06.629427 5780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:46:06.637979 5780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:46:06.638163 5780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:46:06.638386 5780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:46:06.652081 5780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:46:06.653817 5780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:46:06.655344 5780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:46:06.670869 5785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:06.670904 5786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:06.671257 5788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:46:06.673269 5780 server_base.cc:1034] running on GCE node
I20260806 19:46:06.674121 5780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:46:06.675486 5780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:46:06.676784 5780 hybrid_clock.cc:648] HybridClock initialized: now 1786045566676740 us; error 55 us; skew 500 ppm
I20260806 19:46:06.677238 5780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:46:06 dist-test-slave-77v9 krb5kdc[5673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045566, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:46:06.682757 5780 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:46:06.685177 5780 webserver.cc:460] Webserver started at http://127.21.202.65:37395/ using document root <none> and password file <none>
I20260806 19:46:06.686335 5780 fs_manager.cc:359] Metadata directory not provided
I20260806 19:46:06.686748 5780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:46:06.687548 5780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:46:06.691593 5780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "d31ac125bf7d4fcbbb1065eca180cccf"
format_stamp: "Formatted at 2026-08-06 19:46:06 on dist-test-slave-77v9"
server_key: "de83fef3d9da6c352c0205b32a953806"
server_key_iv: "a998c1e1338789ab540e95df9c29d866"
server_key_version: "encryptionkey@0"
I20260806 19:46:06.692482 5780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "d31ac125bf7d4fcbbb1065eca180cccf"
format_stamp: "Formatted at 2026-08-06 19:46:06 on dist-test-slave-77v9"
server_key: "de83fef3d9da6c352c0205b32a953806"
server_key_iv: "a998c1e1338789ab540e95df9c29d866"
server_key_version: "encryptionkey@0"
I20260806 19:46:06.699209 5780 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 19:46:06.703487 5795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:06.705672 5780 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:46:06.705896 5780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "d31ac125bf7d4fcbbb1065eca180cccf"
format_stamp: "Formatted at 2026-08-06 19:46:06 on dist-test-slave-77v9"
server_key: "de83fef3d9da6c352c0205b32a953806"
server_key_iv: "a998c1e1338789ab540e95df9c29d866"
server_key_version: "encryptionkey@0"
I20260806 19:46:06.706102 5780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:46:06.721626 5780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:46:06.728469 5780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:46:06.728988 5780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:46:06.730535 5780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:46:06.732535 5780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:46:06.732635 5780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:06.732790 5780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:46:06.732897 5780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 19:46:06.778717 5907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:41599 every 8 connection(s)
I20260806 19:46:06.778793 5780 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:41599
I20260806 19:46:06.781343 5780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:46:06.790978 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 5780
I20260806 19:46:06.791333 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:46:06.792003 22313 external_mini_cluster.cc:1428] Setting key f4a9d4d9f3f0461f06282f9900bf122c
Aug 06 19:46:06 dist-test-slave-77v9 krb5kdc[5673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045566, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:46:06.813504 5910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:06.786153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:57781 (local address 127.21.202.126:46849)
0806 19:46:06.786726 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:06.786730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:06.788374 (+ 1644us) server_negotiation.cc:407] Connection header received
0806 19:46:06.790666 (+ 2292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:06.790672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:06.790784 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:06.790916 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:06.795200 (+ 4284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:06.795988 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:06.797907 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:06.798323 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:06.798508 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:06.803835 (+ 5327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:06.803847 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:06.803855 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:06.803919 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:06.808511 (+ 4592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:06.810008 (+ 1497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:06.810012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:06.810015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:06.810159 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:06.811203 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:06.811207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:06.811210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:06.811555 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:06.811699 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:06.813006 (+ 1307us) server_negotiation.cc:299] Negotiation successful
0806 19:46:06.813213 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":331,"threads_started":1}
I20260806 19:46:06.815043 5911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:06.787233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:57781)
0806 19:46:06.788146 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:06.788203 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:06.790280 (+ 2077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:06.791378 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:06.791389 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:06.793251 (+ 1862us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:46:06.794955 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:06.794993 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:06.796192 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:06.796196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:06.797702 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:06.797718 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:06.798612 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:06.798622 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:06.798917 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:06.799955 (+ 1038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:46:06.799997 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:46:06.803434 (+ 3437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:46:06.808785 (+ 5351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:46:06.808864 (+ 79us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:46:06.808889 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:46:06.809727 (+ 838us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:46:06.810449 (+ 722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:46:06.810460 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:46:06.810467 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:46:06.811022 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:46:06.811723 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:46:06.811735 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:46:06.812264 (+ 529us) client_negotiation.cc:769] Sending connection context
0806 19:46:06.812592 (+ 328us) client_negotiation.cc:240] Negotiation successful
0806 19:46:06.813335 (+ 743us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":378,"threads_started":1}
I20260806 19:46:06.817574 5908 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46849
I20260806 19:46:06.817945 5908 heartbeater.cc:464] Registering TS with master...
I20260806 19:46:06.818797 5908 heartbeater.cc:511] Master 127.21.202.126:46849 requested a full tablet report, sending...
I20260806 19:46:06.821588 5718 ts_manager.cc:194] Registered new tserver with Master: d31ac125bf7d4fcbbb1065eca180cccf (127.21.202.65:41599)
I20260806 19:46:06.823987 5718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:57781
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:46:06.881860 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46849
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:46:07.287606 5914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:46:07.287982 5914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:46:07.288126 5914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:46:07.296334 5914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:46:07.296504 5914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:46:07.296757 5914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:46:07.309480 5914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:46:07.311930 5914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:46:07.314221 5914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:46:07.330135 5920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:07.330108 5919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:07.330317 5923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:46:07.331535 5914 server_base.cc:1034] running on GCE node
I20260806 19:46:07.332297 5914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:46:07.333537 5914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:46:07.334954 5914 hybrid_clock.cc:648] HybridClock initialized: now 1786045567334824 us; error 61 us; skew 500 ppm
I20260806 19:46:07.335711 5914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:46:07 dist-test-slave-77v9 krb5kdc[5673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045567, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:46:07.340572 5914 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:46:07.342990 5914 webserver.cc:460] Webserver started at http://127.21.202.66:35831/ using document root <none> and password file <none>
I20260806 19:46:07.343905 5914 fs_manager.cc:359] Metadata directory not provided
I20260806 19:46:07.344091 5914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:46:07.344450 5914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:46:07.349574 5914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "061b183a9334432699e36f839ab4e4e3"
format_stamp: "Formatted at 2026-08-06 19:46:07 on dist-test-slave-77v9"
server_key: "4bf1a529ac8ac3839990035dff45d168"
server_key_iv: "dbf97b53c5e98187081acc34850e2015"
server_key_version: "encryptionkey@0"
I20260806 19:46:07.350659 5914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "061b183a9334432699e36f839ab4e4e3"
format_stamp: "Formatted at 2026-08-06 19:46:07 on dist-test-slave-77v9"
server_key: "4bf1a529ac8ac3839990035dff45d168"
server_key_iv: "dbf97b53c5e98187081acc34850e2015"
server_key_version: "encryptionkey@0"
I20260806 19:46:07.358180 5914 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260806 19:46:07.364099 5930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:07.365828 5914 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 19:46:07.366133 5914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "061b183a9334432699e36f839ab4e4e3"
format_stamp: "Formatted at 2026-08-06 19:46:07 on dist-test-slave-77v9"
server_key: "4bf1a529ac8ac3839990035dff45d168"
server_key_iv: "dbf97b53c5e98187081acc34850e2015"
server_key_version: "encryptionkey@0"
I20260806 19:46:07.366380 5914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:46:07.395961 5914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:46:07.401183 5914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:46:07.401532 5914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:46:07.402621 5914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:46:07.404179 5914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:46:07.404289 5914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:07.404458 5914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:46:07.404538 5914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:07.450798 5914 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:33417
I20260806 19:46:07.450901 6042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:33417 every 8 connection(s)
I20260806 19:46:07.452157 5914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:46:07.455011 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 5914
I20260806 19:46:07.455250 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:46:07.455796 22313 external_mini_cluster.cc:1428] Setting key 61db8f0386a0e9a9b3ba2977d56ffb42
Aug 06 19:46:07 dist-test-slave-77v9 krb5kdc[5673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045567, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:46:07.480094 6045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:07.455080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:59011 (local address 127.21.202.126:46849)
0806 19:46:07.455623 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:07.455629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:07.456891 (+ 1262us) server_negotiation.cc:407] Connection header received
0806 19:46:07.459648 (+ 2757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:07.459652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:07.459720 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:07.459864 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:07.462545 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:07.463493 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:07.465396 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:07.465913 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:07.466067 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:07.471130 (+ 5063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:07.471139 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:07.471148 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:07.471206 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:07.475338 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:07.476678 (+ 1340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:07.476682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:07.476685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:07.477010 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:07.477789 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:07.477794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:07.477796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:07.478171 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:07.478328 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:07.479477 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0806 19:46:07.479759 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":274,"threads_started":1}
I20260806 19:46:07.481838 6046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:07.455631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:59011)
0806 19:46:07.456600 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:07.456652 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:07.459113 (+ 2461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:07.460161 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:07.460178 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:07.461163 (+ 985us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:46:07.462251 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:07.462282 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:07.463699 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:07.463711 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:07.465209 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:07.465224 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:07.466194 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:07.466202 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:07.466486 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:07.467277 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:46:07.467302 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:46:07.470644 (+ 3342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:46:07.475583 (+ 4939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:46:07.475598 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:46:07.475624 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:46:07.476445 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:46:07.477159 (+ 714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:46:07.477167 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:46:07.477173 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:46:07.477481 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:46:07.478335 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:46:07.478347 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:46:07.478903 (+ 556us) client_negotiation.cc:769] Sending connection context
0806 19:46:07.479282 (+ 379us) client_negotiation.cc:240] Negotiation successful
0806 19:46:07.479944 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":336,"threads_started":1}
I20260806 19:46:07.483995 6043 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46849
I20260806 19:46:07.484345 6043 heartbeater.cc:464] Registering TS with master...
I20260806 19:46:07.485212 6043 heartbeater.cc:511] Master 127.21.202.126:46849 requested a full tablet report, sending...
I20260806 19:46:07.487192 5718 ts_manager.cc:194] Registered new tserver with Master: 061b183a9334432699e36f839ab4e4e3 (127.21.202.66:33417)
I20260806 19:46:07.488085 5718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:59011
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:46:07.537918 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46849
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:46:07.829722 5908 heartbeater.cc:503] Master 127.21.202.126:46849 was elected leader, sending a full tablet report...
W20260806 19:46:07.961215 6049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:46:07.961560 6049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:46:07.961706 6049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:46:07.970377 6049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:46:07.970598 6049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:46:07.970800 6049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:46:07.982952 6049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:46:07.984683 6049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:46:07.986351 6049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:46:08.004407 6054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:08.004478 6055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:08.004506 6057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:46:08.004901 6049 server_base.cc:1034] running on GCE node
I20260806 19:46:08.005998 6049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:46:08.007498 6049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:46:08.008761 6049 hybrid_clock.cc:648] HybridClock initialized: now 1786045568008706 us; error 58 us; skew 500 ppm
I20260806 19:46:08.009233 6049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:46:08 dist-test-slave-77v9 krb5kdc[5673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045568, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:46:08.013096 6049 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:46:08.014767 6049 webserver.cc:460] Webserver started at http://127.21.202.67:38747/ using document root <none> and password file <none>
I20260806 19:46:08.015573 6049 fs_manager.cc:359] Metadata directory not provided
I20260806 19:46:08.015686 6049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:46:08.015945 6049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:46:08.019167 6049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "6c0836279c1749fda03a4d41ad11588a"
format_stamp: "Formatted at 2026-08-06 19:46:08 on dist-test-slave-77v9"
server_key: "a83a9c314f97581dddf9b3cb18f2f9bd"
server_key_iv: "398ab31fbb63d3b65ca08011b047a288"
server_key_version: "encryptionkey@0"
I20260806 19:46:08.019949 6049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "6c0836279c1749fda03a4d41ad11588a"
format_stamp: "Formatted at 2026-08-06 19:46:08 on dist-test-slave-77v9"
server_key: "a83a9c314f97581dddf9b3cb18f2f9bd"
server_key_iv: "398ab31fbb63d3b65ca08011b047a288"
server_key_version: "encryptionkey@0"
I20260806 19:46:08.026322 6049 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 19:46:08.031548 6065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:08.033594 6049 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 19:46:08.033813 6049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "6c0836279c1749fda03a4d41ad11588a"
format_stamp: "Formatted at 2026-08-06 19:46:08 on dist-test-slave-77v9"
server_key: "a83a9c314f97581dddf9b3cb18f2f9bd"
server_key_iv: "398ab31fbb63d3b65ca08011b047a288"
server_key_version: "encryptionkey@0"
I20260806 19:46:08.034068 6049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:46:08.057560 6049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:46:08.063337 6049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:46:08.063638 6049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:46:08.064980 6049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:46:08.066550 6049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:46:08.066635 6049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:08.066794 6049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:46:08.066864 6049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:08.116889 6049 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:39295
I20260806 19:46:08.117252 6177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:39295 every 8 connection(s)
I20260806 19:46:08.118920 6049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:46:08.128337 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 6049
I20260806 19:46:08.128613 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:46:08.129185 22313 external_mini_cluster.cc:1428] Setting key 8210b61b65bd7237f7d399e132d8d397
Aug 06 19:46:08 dist-test-slave-77v9 krb5kdc[5673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045568, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:46:08.147917 6180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.122845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:37635 (local address 127.21.202.126:46849)
0806 19:46:08.125091 (+ 2246us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:08.125096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:08.125483 (+ 387us) server_negotiation.cc:407] Connection header received
0806 19:46:08.127570 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:08.127575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:08.127650 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:08.127765 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:08.130636 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.132025 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.135138 (+ 3113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.135580 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.135750 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.140753 (+ 5003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:08.140760 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:08.140764 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:08.140933 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:08.143909 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:08.145210 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:08.145216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:08.145219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:08.145406 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:08.146001 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:08.146005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:08.146007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:08.146417 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:08.146562 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:08.147379 (+ 817us) server_negotiation.cc:299] Negotiation successful
0806 19:46:08.147568 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2119,"thread_start_us":1935,"threads_started":1}
I20260806 19:46:08.149026 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.123263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:37635)
0806 19:46:08.125246 (+ 1983us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:08.125289 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:08.127216 (+ 1927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:08.128072 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:08.128082 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:08.129066 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:46:08.130415 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:08.130441 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.132223 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.132229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:08.134902 (+ 2673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:08.134923 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.135881 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.135892 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:08.136184 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.137213 (+ 1029us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:46:08.137245 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:46:08.140338 (+ 3093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:46:08.144142 (+ 3804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:46:08.144156 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:46:08.144179 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:46:08.144795 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:46:08.145572 (+ 777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:46:08.145578 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:46:08.145582 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:46:08.145849 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:46:08.146579 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:46:08.146587 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:46:08.146973 (+ 386us) client_negotiation.cc:769] Sending connection context
0806 19:46:08.147222 (+ 249us) client_negotiation.cc:240] Negotiation successful
0806 19:46:08.147668 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1122,"thread_start_us":673,"threads_started":1}
I20260806 19:46:08.150790 6178 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46849
I20260806 19:46:08.151119 6178 heartbeater.cc:464] Registering TS with master...
I20260806 19:46:08.151921 6178 heartbeater.cc:511] Master 127.21.202.126:46849 requested a full tablet report, sending...
I20260806 19:46:08.153596 5718 ts_manager.cc:194] Registered new tserver with Master: 6c0836279c1749fda03a4d41ad11588a (127.21.202.67:39295)
I20260806 19:46:08.154520 5718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:37635
I20260806 19:46:08.161053 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:46:08.188769 6180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.167920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49360 (local address 127.21.202.126:46849)
0806 19:46:08.168317 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:08.168325 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:08.168406 (+ 81us) server_negotiation.cc:407] Connection header received
0806 19:46:08.168661 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:08.168672 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:08.168920 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:08.169023 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:08.171028 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.172199 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.174293 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.175142 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.175274 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.178530 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:08.178539 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:08.178544 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:08.178811 (+ 267us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:08.183311 (+ 4500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:08.184427 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:08.184432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:08.184435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:08.184719 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:08.185599 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:08.185604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:08.185606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:08.186466 (+ 860us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:08.186740 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:08.187884 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0806 19:46:08.188225 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 19:46:08.194780 5718 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:46:08.202849 5718 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:46:08.246237 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.221802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:33417 (local address 127.0.0.1:37494)
0806 19:46:08.223858 (+ 2056us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:08.223954 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:08.224360 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:08.233013 (+ 8653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:08.233029 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:08.233096 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:46:08.233745 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:08.233764 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.239493 (+ 5729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.239501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:08.242601 (+ 3100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:08.242618 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.245097 (+ 2479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.245104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:08.245344 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.245410 (+ 66us) client_negotiation.cc:769] Sending connection context
0806 19:46:08.245557 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 19:46:08.245783 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1621,"thread_start_us":1430,"threads_started":1}
I20260806 19:46:08.247488 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.221936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37494 (local address 127.21.202.66:33417)
0806 19:46:08.223298 (+ 1362us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:08.223311 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:08.224698 (+ 1387us) server_negotiation.cc:407] Connection header received
0806 19:46:08.224899 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:08.224915 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:08.225413 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:08.232029 (+ 6616us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:46:08.234586 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.239209 (+ 4623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.242836 (+ 3627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.244517 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.244680 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.244954 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:08.246894 (+ 1940us) server_negotiation.cc:299] Negotiation successful
0806 19:46:08.247126 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1066,"thread_start_us":971,"threads_started":1}
I20260806 19:46:08.250723 6194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.223445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:41599 (local address 127.0.0.1:50906)
0806 19:46:08.240135 (+ 16690us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:08.240157 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:08.240326 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:08.241486 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:08.241491 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:08.241520 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:46:08.242013 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:08.242027 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.244098 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.244103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:08.246934 (+ 2831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:08.246948 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.249829 (+ 2881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.249839 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:08.250038 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.250072 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 19:46:08.250182 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 19:46:08.250357 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16525,"thread_start_us":16313,"threads_started":1}
I20260806 19:46:08.250777 6193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.223420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50906 (local address 127.21.202.65:41599)
0806 19:46:08.238949 (+ 15529us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:08.238964 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:08.240313 (+ 1349us) server_negotiation.cc:407] Connection header received
0806 19:46:08.240574 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:08.240584 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:08.240964 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:08.241217 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:46:08.242254 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.243924 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.247129 (+ 3205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.249264 (+ 2135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.249408 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.249682 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:08.250235 (+ 553us) server_negotiation.cc:299] Negotiation successful
0806 19:46:08.250424 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15191,"thread_start_us":15837,"threads_started":1}
I20260806 19:46:08.253511 5978 tablet_service.cc:1467] Processing CreateTablet for tablet 7cd64e16b0da441d83eec27c051e8957 (DEFAULT_TABLE table=test-table [id=6b6d66719dab4deba919a915ebdef208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:46:08.254431 5843 tablet_service.cc:1467] Processing CreateTablet for tablet 7cd64e16b0da441d83eec27c051e8957 (DEFAULT_TABLE table=test-table [id=6b6d66719dab4deba919a915ebdef208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:46:08.255170 5978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cd64e16b0da441d83eec27c051e8957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:46:08.255782 5843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cd64e16b0da441d83eec27c051e8957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:46:08.258520 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.225349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:39295 (local address 127.0.0.1:42654)
0806 19:46:08.246583 (+ 21234us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:08.246606 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:08.246716 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:08.249646 (+ 2930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:08.249651 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:08.249683 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:46:08.249869 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:08.249880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.253226 (+ 3346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.253232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:08.255150 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:08.255179 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.257658 (+ 2479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.257664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:08.257865 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.257904 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 19:46:08.258000 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 19:46:08.258170 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21129,"thread_start_us":23498,"threads_started":1}
I20260806 19:46:08.258894 6196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.225992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42654 (local address 127.21.202.67:39295)
0806 19:46:08.248654 (+ 22662us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:08.248664 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:08.248702 (+ 38us) server_negotiation.cc:407] Connection header received
0806 19:46:08.248852 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:08.248860 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:08.249178 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:08.249425 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:46:08.250060 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.252962 (+ 2902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.255401 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.257454 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.257611 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.258208 (+ 597us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:08.258288 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 19:46:08.258494 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22438,"thread_start_us":22222,"threads_started":1}
I20260806 19:46:08.263733 6113 tablet_service.cc:1467] Processing CreateTablet for tablet 7cd64e16b0da441d83eec27c051e8957 (DEFAULT_TABLE table=test-table [id=6b6d66719dab4deba919a915ebdef208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:46:08.265734 6113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cd64e16b0da441d83eec27c051e8957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:46:08.270808 6197 tablet_bootstrap.cc:492] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3: Bootstrap starting.
I20260806 19:46:08.273960 6198 tablet_bootstrap.cc:492] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf: Bootstrap starting.
I20260806 19:46:08.275554 6197 tablet_bootstrap.cc:654] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3: Neither blocks nor log segments found. Creating new log.
I20260806 19:46:08.277125 6197 log.cc:824] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3: Log is configured to *not* fsync() on all Append() calls
I20260806 19:46:08.278311 6198 tablet_bootstrap.cc:654] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf: Neither blocks nor log segments found. Creating new log.
I20260806 19:46:08.281332 6198 log.cc:824] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf: Log is configured to *not* fsync() on all Append() calls
I20260806 19:46:08.282107 6197 tablet_bootstrap.cc:492] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3: No bootstrap required, opened a new log
I20260806 19:46:08.282528 6197 ts_tablet_manager.cc:1397] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.008s
I20260806 19:46:08.287461 6201 tablet_bootstrap.cc:492] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a: Bootstrap starting.
I20260806 19:46:08.290792 6201 tablet_bootstrap.cc:654] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a: Neither blocks nor log segments found. Creating new log.
I20260806 19:46:08.287205 6198 tablet_bootstrap.cc:492] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf: No bootstrap required, opened a new log
I20260806 19:46:08.292045 6198 ts_tablet_manager.cc:1397] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.003s
I20260806 19:46:08.292328 6201 log.cc:824] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:46:08.293123 6197 raft_consensus.cc:348] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:08.293663 6197 raft_consensus.cc:374] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:46:08.293807 6197 raft_consensus.cc:729] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 061b183a9334432699e36f839ab4e4e3, State: Initialized, Role: FOLLOWER
I20260806 19:46:08.294505 6197 consensus_queue.cc:260] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:08.295740 6043 heartbeater.cc:503] Master 127.21.202.126:46849 was elected leader, sending a full tablet report...
I20260806 19:46:08.295965 6201 tablet_bootstrap.cc:492] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a: No bootstrap required, opened a new log
I20260806 19:46:08.296276 6201 ts_tablet_manager.cc:1397] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.006s
I20260806 19:46:08.296494 6197 ts_tablet_manager.cc:1428] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3: Time spent starting tablet: real 0.014s user 0.007s sys 0.006s
I20260806 19:46:08.303094 6198 raft_consensus.cc:348] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:08.303839 6198 raft_consensus.cc:374] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:46:08.304003 6198 raft_consensus.cc:729] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d31ac125bf7d4fcbbb1065eca180cccf, State: Initialized, Role: FOLLOWER
I20260806 19:46:08.304800 6198 consensus_queue.cc:260] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:08.307255 6198 ts_tablet_manager.cc:1428] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf: Time spent starting tablet: real 0.015s user 0.007s sys 0.007s
I20260806 19:46:08.307910 6201 raft_consensus.cc:348] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:08.308351 6201 raft_consensus.cc:374] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:46:08.308478 6201 raft_consensus.cc:729] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c0836279c1749fda03a4d41ad11588a, State: Initialized, Role: FOLLOWER
I20260806 19:46:08.309149 6201 consensus_queue.cc:260] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:08.310353 6178 heartbeater.cc:503] Master 127.21.202.126:46849 was elected leader, sending a full tablet report...
I20260806 19:46:08.311264 6201 ts_tablet_manager.cc:1428] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
W20260806 19:46:08.371304 6179 tablet.cc:2369] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:46:08.454704 6044 tablet.cc:2369] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:46:08.490670 6203 raft_consensus.cc:482] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:46:08.491137 6203 raft_consensus.cc:504] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:08.493472 6203 leader_election.cc:288] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c0836279c1749fda03a4d41ad11588a (127.21.202.67:39295), d31ac125bf7d4fcbbb1065eca180cccf (127.21.202.65:41599)
I20260806 19:46:08.503291 6196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.494118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:39659 (local address 127.21.202.67:39295)
0806 19:46:08.494417 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:08.494427 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:08.495137 (+ 710us) server_negotiation.cc:407] Connection header received
0806 19:46:08.495390 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:08.495396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:08.495484 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:08.495598 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:46:08.496234 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.497468 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.500587 (+ 3119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.501991 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.502170 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.502254 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:08.502856 (+ 602us) server_negotiation.cc:299] Negotiation successful
0806 19:46:08.503052 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 19:46:08.503285 6193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.494026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:46473 (local address 127.21.202.65:41599)
0806 19:46:08.494321 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:08.494330 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:08.495255 (+ 925us) server_negotiation.cc:407] Connection header received
0806 19:46:08.495505 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:08.495510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:08.495615 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:08.495703 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:46:08.496666 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.497731 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.500588 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.501967 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.502170 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.502254 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:08.502856 (+ 602us) server_negotiation.cc:299] Negotiation successful
0806 19:46:08.503005 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 19:46:08.503996 6207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.493913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:39295 (local address 127.21.202.66:39659)
0806 19:46:08.494960 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:08.494983 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:08.495180 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:08.495623 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:08.495627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:08.495656 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:46:08.496077 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:08.496087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.497641 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.497645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:08.500285 (+ 2640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:08.500306 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.502307 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.502315 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:08.502563 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.502644 (+ 81us) client_negotiation.cc:769] Sending connection context
0806 19:46:08.502765 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 19:46:08.503162 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":183,"thread_start_us":720,"threads_started":1}
I20260806 19:46:08.504097 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.493957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:41599 (local address 127.21.202.66:46473)
0806 19:46:08.495165 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:08.495181 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:08.495306 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:08.495802 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:08.495807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:08.495919 (+ 112us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:46:08.496453 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:08.496466 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.497871 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.497880 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:08.500289 (+ 2409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:08.500306 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.502322 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.502326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:08.502606 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.502653 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 19:46:08.502764 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 19:46:08.503162 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"mutex_wait_us":284,"spinlock_wait_cycles":26880,"thread_start_us":495,"threads_started":1}
I20260806 19:46:08.505425 6133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cd64e16b0da441d83eec27c051e8957" candidate_uuid: "061b183a9334432699e36f839ab4e4e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c0836279c1749fda03a4d41ad11588a" is_pre_election: true
I20260806 19:46:08.505405 5863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cd64e16b0da441d83eec27c051e8957" candidate_uuid: "061b183a9334432699e36f839ab4e4e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" is_pre_election: true
I20260806 19:46:08.506057 6133 raft_consensus.cc:2454] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 061b183a9334432699e36f839ab4e4e3 in term 0.
I20260806 19:46:08.506067 5863 raft_consensus.cc:2454] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 061b183a9334432699e36f839ab4e4e3 in term 0.
I20260806 19:46:08.506937 5931 leader_election.cc:302] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 061b183a9334432699e36f839ab4e4e3, 6c0836279c1749fda03a4d41ad11588a, d31ac125bf7d4fcbbb1065eca180cccf; no voters:
I20260806 19:46:08.507579 6203 raft_consensus.cc:2781] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:46:08.507762 6203 raft_consensus.cc:482] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:46:08.507880 6203 raft_consensus.cc:3037] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:46:08.510527 6203 raft_consensus.cc:504] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:08.511245 6203 leader_election.cc:288] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [CANDIDATE]: Term 1 election: Requested vote from peers 6c0836279c1749fda03a4d41ad11588a (127.21.202.67:39295), d31ac125bf7d4fcbbb1065eca180cccf (127.21.202.65:41599)
I20260806 19:46:08.511955 5863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cd64e16b0da441d83eec27c051e8957" candidate_uuid: "061b183a9334432699e36f839ab4e4e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d31ac125bf7d4fcbbb1065eca180cccf"
I20260806 19:46:08.512228 5863 raft_consensus.cc:3037] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:46:08.512230 6133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cd64e16b0da441d83eec27c051e8957" candidate_uuid: "061b183a9334432699e36f839ab4e4e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c0836279c1749fda03a4d41ad11588a"
I20260806 19:46:08.512526 6133 raft_consensus.cc:3037] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:46:08.514787 5863 raft_consensus.cc:2454] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 061b183a9334432699e36f839ab4e4e3 in term 1.
I20260806 19:46:08.514881 6133 raft_consensus.cc:2454] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 061b183a9334432699e36f839ab4e4e3 in term 1.
I20260806 19:46:08.515363 5934 leader_election.cc:302] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 061b183a9334432699e36f839ab4e4e3, d31ac125bf7d4fcbbb1065eca180cccf; no voters:
I20260806 19:46:08.515789 6203 raft_consensus.cc:2781] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:46:08.516366 6203 raft_consensus.cc:686] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [term 1 LEADER]: Becoming Leader. State: Replica: 061b183a9334432699e36f839ab4e4e3, State: Running, Role: LEADER
I20260806 19:46:08.516979 6203 consensus_queue.cc:237] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:08.522646 5717 catalog_manager.cc:5495] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 061b183a9334432699e36f839ab4e4e3 (127.21.202.66). New cstate: current_term: 1 leader_uuid: "061b183a9334432699e36f839ab4e4e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } health_report { overall_health: UNKNOWN } } }
W20260806 19:46:08.535085 5909 tablet.cc:2369] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:46:08.569512 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.563493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37504 (local address 127.21.202.66:33417)
0806 19:46:08.563704 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:08.563711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:08.563733 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:46:08.563953 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:08.563957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:08.564038 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:08.564209 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:46:08.564724 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.565545 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.567078 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.567537 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.567679 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.568250 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:46:08.568787 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:46:08.569055 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:08.569111 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 19:46:08.569201 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 19:46:08.579672 5863 raft_consensus.cc:1261] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf [term 1 FOLLOWER]: Refusing update from remote peer 061b183a9334432699e36f839ab4e4e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:46:08.579671 6133 raft_consensus.cc:1261] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a [term 1 FOLLOWER]: Refusing update from remote peer 061b183a9334432699e36f839ab4e4e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:46:08.581250 6208 consensus_queue.cc:1035] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 19:46:08.581735 6203 consensus_queue.cc:1035] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:46:08.608291 6212 mvcc.cc:205] Tried to move back new op lower bound from 7315642648889171968 to 7315642648650145792. Current Snapshot: MvccSnapshot[applied={T|T < 7315642648889171968}]
I20260806 19:46:08.620323 5717 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:46:08.620736 5717 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:46:08.625309 5717 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Sending DeleteTablet for 3 replicas of tablet 7cd64e16b0da441d83eec27c051e8957
I20260806 19:46:08.626950 5843 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cd64e16b0da441d83eec27c051e8957 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:46:08 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:50906
I20260806 19:46:08.627465 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 5780
I20260806 19:46:08.627381 6113 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cd64e16b0da441d83eec27c051e8957 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:46:08 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:42654
I20260806 19:46:08.628396 5978 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cd64e16b0da441d83eec27c051e8957 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:46:08 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:37494
I20260806 19:46:08.639690 6221 tablet_replica.cc:330] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf: stopping tablet replica
I20260806 19:46:08.640487 6221 raft_consensus.cc:2229] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:46:08.641203 6221 raft_consensus.cc:2258] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:46:08.643767 6221 ts_tablet_manager.cc:1905] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:46:08.651881 6221 ts_tablet_manager.cc:1918] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:46:08.652236 6221 log.cc:1196] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/7cd64e16b0da441d83eec27c051e8957
I20260806 19:46:08.653005 6221 ts_tablet_manager.cc:1939] T 7cd64e16b0da441d83eec27c051e8957 P d31ac125bf7d4fcbbb1065eca180cccf: Deleting consensus metadata
I20260806 19:46:08.654042 6224 tablet_replica.cc:330] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3: stopping tablet replica
I20260806 19:46:08.655015 6224 raft_consensus.cc:2229] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:46:08.655450 5705 catalog_manager.cc:4841] TS d31ac125bf7d4fcbbb1065eca180cccf (127.21.202.65:41599): tablet 7cd64e16b0da441d83eec27c051e8957 (table test-table [id=6b6d66719dab4deba919a915ebdef208]) successfully deleted
I20260806 19:46:08.657089 6222 tablet_replica.cc:330] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a: stopping tablet replica
I20260806 19:46:08.657204 6224 raft_consensus.cc:2258] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:46:08.657691 6222 raft_consensus.cc:2229] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:46:08.658313 6222 raft_consensus.cc:2258] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:46:08.660128 6224 ts_tablet_manager.cc:1905] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:46:08.660434 6222 ts_tablet_manager.cc:1905] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:46:08.668221 6224 ts_tablet_manager.cc:1918] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:46:08.668478 6224 log.cc:1196] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/7cd64e16b0da441d83eec27c051e8957
I20260806 19:46:08.668707 6222 ts_tablet_manager.cc:1918] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:46:08.668944 6222 log.cc:1196] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/7cd64e16b0da441d83eec27c051e8957
I20260806 19:46:08.669164 6224 ts_tablet_manager.cc:1939] T 7cd64e16b0da441d83eec27c051e8957 P 061b183a9334432699e36f839ab4e4e3: Deleting consensus metadata
I20260806 19:46:08.669732 6222 ts_tablet_manager.cc:1939] T 7cd64e16b0da441d83eec27c051e8957 P 6c0836279c1749fda03a4d41ad11588a: Deleting consensus metadata
I20260806 19:46:08.672060 5702 catalog_manager.cc:4841] TS 061b183a9334432699e36f839ab4e4e3 (127.21.202.66:33417): tablet 7cd64e16b0da441d83eec27c051e8957 (table test-table [id=6b6d66719dab4deba919a915ebdef208]) successfully deleted
I20260806 19:46:08.673470 5702 catalog_manager.cc:4841] TS 6c0836279c1749fda03a4d41ad11588a (127.21.202.67:39295): tablet 7cd64e16b0da441d83eec27c051e8957 (table test-table [id=6b6d66719dab4deba919a915ebdef208]) successfully deleted
Aug 06 19:46:08 dist-test-slave-77v9 krb5kdc[5673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:46:08.684391 6193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.629130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50916 (local address 127.21.202.65:41599)
0806 19:46:08.629386 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:08.629395 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:08.656870 (+ 27475us) server_negotiation.cc:407] Connection header received
0806 19:46:08.657142 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:08.657150 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:08.657256 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:08.657372 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:08.660737 (+ 3365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.661951 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.664326 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.665022 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.665139 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.675184 (+ 10045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:08.675212 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:08.675233 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:08.675309 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:08.680009 (+ 4700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:08.681225 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:08.681240 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:08.681244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:08.681725 (+ 481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:08.682285 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:08.682293 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:08.682298 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:08.682924 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:08.683249 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:08.683695 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 19:46:08.683952 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 19:46:08.685200 5903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:46:08.792349 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 5780
I20260806 19:46:08.820899 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 5914
Aug 06 19:46:08 dist-test-slave-77v9 krb5kdc[5673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:46:08.843425 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:08.821818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58926 (local address 127.21.202.66:33417)
0806 19:46:08.821989 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:08.821993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:08.822010 (+ 17us) server_negotiation.cc:407] Connection header received
0806 19:46:08.822263 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:08.822268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:08.822348 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:08.822461 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:08.824268 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.825535 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.828734 (+ 3199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:08.829342 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:08.829463 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:08.835648 (+ 6185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:08.835670 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:08.835690 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:08.835753 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:08.839670 (+ 3917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:08.840756 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:08.840765 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:08.840768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:08.841029 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:08.841518 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:08.841524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:08.841530 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:08.841951 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:08.842359 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:08.842702 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 19:46:08.842979 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 19:46:08.844300 6038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:46:08.976326 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 5914
I20260806 19:46:09.000902 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 6049
Aug 06 19:46:09 dist-test-slave-77v9 krb5kdc[5673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:46:09.024066 6196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:09.001861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56904 (local address 127.21.202.67:39295)
0806 19:46:09.002289 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:09.002298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:09.002457 (+ 159us) server_negotiation.cc:407] Connection header received
0806 19:46:09.002802 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:09.002808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:09.002897 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:09.003031 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:09.005311 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:09.006107 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:09.007586 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:09.008003 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:09.008153 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:09.016193 (+ 8040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:09.016214 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:09.016231 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:09.016284 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:09.020295 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:09.021546 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:09.021554 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:09.021556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:09.021747 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:09.022307 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:09.022312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:09.022314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:09.022772 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:09.023090 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:09.023435 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 19:46:09.023638 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260806 19:46:09.025406 6173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:46:09.132390 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 6049
I20260806 19:46:09.154680 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 5686
I20260806 19:46:09.156108 5748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:46:09.277427 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 5686
I20260806 19:46:29.304946 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46849
--webserver_interface=127.21.202.126
--webserver_port=44555
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:46849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:46:29.706439 6229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:46:29.706775 6229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:46:29.706934 6229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:46:29.714640 6229 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 19:46:29.714759 6229 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 19:46:29.714799 6229 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 19:46:29.714913 6229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:46:29.715008 6229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:46:29.715131 6229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:46:29.715188 6229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:46:29.715238 6229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:46:29.726790 6229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:46849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=44555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:46:29.728274 6229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:46:29.729887 6229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:46:29.758355 6235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:29.758422 6238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:29.758283 6236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:46:29.759083 6229 server_base.cc:1034] running on GCE node
I20260806 19:46:29.760577 6229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:46:29.763862 6229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:46:29.765331 6229 hybrid_clock.cc:648] HybridClock initialized: now 1786045589765198 us; error 62 us; skew 500 ppm
I20260806 19:46:29.766098 6229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:46:29 dist-test-slave-77v9 krb5kdc[5673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045589, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:46:29.771224 6229 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:46:29.773458 6229 webserver.cc:460] Webserver started at http://127.21.202.126:44555/ using document root <none> and password file <none>
I20260806 19:46:29.774320 6229 fs_manager.cc:359] Metadata directory not provided
I20260806 19:46:29.774443 6229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:46:29.782292 6229 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:46:29.786527 6245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:29.788434 6229 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260806 19:46:29.788671 6229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "7d4dc71690334648b218d29a91b1335a"
format_stamp: "Formatted at 2026-08-06 19:46:05 on dist-test-slave-77v9"
server_key: "8c892914178612f5253e810bc6aa7086"
server_key_iv: "20462370340cc7784d3c98aab4fa70e8"
server_key_version: "encryptionkey@0"
I20260806 19:46:29.789918 6229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:46:29.803746 6229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:46:29.808768 6229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:46:29.809182 6229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:46:29.832036 6229 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:46849
I20260806 19:46:29.832077 6296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:46849 every 8 connection(s)
I20260806 19:46:29.833899 6229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:46:29.842309 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 6229
I20260806 19:46:29.843083 6297 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:46:29.843479 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:41599
--local_ip_for_outbound_sockets=127.21.202.65
--tserver_master_addrs=127.21.202.126:46849
--webserver_port=37395
--webserver_interface=127.21.202.65
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:46:29.848927 6297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Bootstrap starting.
I20260806 19:46:29.875164 6297 log.cc:824] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:46:29.889397 6297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:46:29.890026 6297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Bootstrap complete.
I20260806 19:46:29.901688 6297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } }
I20260806 19:46:29.902711 6297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d4dc71690334648b218d29a91b1335a, State: Initialized, Role: FOLLOWER
I20260806 19:46:29.903559 6297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } }
I20260806 19:46:29.903838 6297 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:46:29.903995 6297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:46:29.904210 6297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 1 FOLLOWER]: Advancing to term 2
I20260806 19:46:29.909097 6297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } }
I20260806 19:46:29.909739 6297 leader_election.cc:302] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d4dc71690334648b218d29a91b1335a; no voters:
I20260806 19:46:29.910579 6297 leader_election.cc:288] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 19:46:29.910789 6301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 2 FOLLOWER]: Leader election won for term 2
I20260806 19:46:29.912302 6301 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 2 LEADER]: Becoming Leader. State: Replica: 7d4dc71690334648b218d29a91b1335a, State: Running, Role: LEADER
I20260806 19:46:29.913025 6301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } }
I20260806 19:46:29.913393 6297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:46:29.916940 6303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d4dc71690334648b218d29a91b1335a. Latest consensus state: current_term: 2 leader_uuid: "7d4dc71690334648b218d29a91b1335a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } } }
I20260806 19:46:29.917024 6302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7d4dc71690334648b218d29a91b1335a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } } }
I20260806 19:46:29.918079 6303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: This master's current role is: LEADER
I20260806 19:46:29.918201 6302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: This master's current role is: LEADER
I20260806 19:46:29.929912 6307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:46:29.940299 6307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:46:29.942281 6307 catalog_manager.cc:1259] Loaded cluster ID: e038041afe914edc8f292fb2336fcf0a
I20260806 19:46:29.942495 6307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:46:29.947209 6307 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:46:29.949059 6307 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Loaded TSK: 0
I20260806 19:46:29.983179 6307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Generated new TSK 1
I20260806 19:46:29.983759 6307 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 19:46:30.281177 6299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:46:30.281555 6299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:46:30.281721 6299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:46:30.290427 6299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:46:30.290642 6299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:46:30.290925 6299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:46:30.302855 6299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:41599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=37395
--tserver_master_addrs=127.21.202.126:46849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:46:30.304421 6299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:46:30.305927 6299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:46:30.322387 6324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:30.322635 6325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:30.323957 6327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:46:30.324179 6299 server_base.cc:1034] running on GCE node
I20260806 19:46:30.325093 6299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:46:30.326252 6299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:46:30.327543 6299 hybrid_clock.cc:648] HybridClock initialized: now 1786045590327489 us; error 49 us; skew 500 ppm
I20260806 19:46:30.327939 6299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:46:30 dist-test-slave-77v9 krb5kdc[5673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045590, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:46:30.331832 6299 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:46:30.334448 6299 webserver.cc:460] Webserver started at http://127.21.202.65:37395/ using document root <none> and password file <none>
I20260806 19:46:30.335263 6299 fs_manager.cc:359] Metadata directory not provided
I20260806 19:46:30.335465 6299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:46:30.343022 6299 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:46:30.346544 6334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:30.348609 6299 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:46:30.349036 6299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "d31ac125bf7d4fcbbb1065eca180cccf"
format_stamp: "Formatted at 2026-08-06 19:46:06 on dist-test-slave-77v9"
server_key: "de83fef3d9da6c352c0205b32a953806"
server_key_iv: "a998c1e1338789ab540e95df9c29d866"
server_key_version: "encryptionkey@0"
I20260806 19:46:30.350947 6299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:46:30.375839 6299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:46:30.381044 6299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:46:30.381348 6299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:46:30.382426 6299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:46:30.383910 6299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:46:30.384001 6299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:30.384079 6299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:46:30.384155 6299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:30.426889 6299 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:41599
I20260806 19:46:30.427031 6448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:41599 every 8 connection(s)
I20260806 19:46:30.428470 6299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:46:30.437534 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 6299
I20260806 19:46:30.438493 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:33417
--local_ip_for_outbound_sockets=127.21.202.66
--tserver_master_addrs=127.21.202.126:46849
--webserver_port=35831
--webserver_interface=127.21.202.66
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:46:30 dist-test-slave-77v9 krb5kdc[5673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045590, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:46:30.459841 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:30.431790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:55891 (local address 127.21.202.126:46849)
0806 19:46:30.432541 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:30.432553 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:30.433183 (+ 630us) server_negotiation.cc:407] Connection header received
0806 19:46:30.435408 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:30.435419 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:30.435750 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:30.436063 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:30.437954 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:30.440012 (+ 2058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:30.442101 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:30.443235 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:30.443474 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:30.449163 (+ 5689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:30.449179 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:30.449208 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:30.449287 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:30.453855 (+ 4568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:30.455185 (+ 1330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:30.455195 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:30.455201 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:30.455402 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:30.456153 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:30.456161 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:30.456168 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:30.457133 (+ 965us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:30.457324 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:30.458409 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0806 19:46:30.458824 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":281,"threads_started":1}
I20260806 19:46:30.460235 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:30.432059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:55891)
0806 19:46:30.432953 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:30.433012 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:30.434832 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:30.436284 (+ 1452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:30.436292 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:30.437020 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:46:30.437633 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:30.437653 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:30.440318 (+ 2665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:30.440324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:30.441924 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:30.441940 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:30.443751 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:30.443763 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:30.444066 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:30.445065 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:46:30.445099 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:46:30.448513 (+ 3414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:46:30.454090 (+ 5577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:46:30.454107 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:46:30.454128 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:46:30.454924 (+ 796us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:46:30.455587 (+ 663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:46:30.455594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:46:30.455601 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:46:30.455906 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:46:30.457289 (+ 1383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:46:30.457301 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:46:30.457810 (+ 509us) client_negotiation.cc:769] Sending connection context
0806 19:46:30.458170 (+ 360us) client_negotiation.cc:240] Negotiation successful
0806 19:46:30.458820 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":382,"threads_started":1}
I20260806 19:46:30.463184 6449 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46849
I20260806 19:46:30.463533 6449 heartbeater.cc:464] Registering TS with master...
I20260806 19:46:30.464597 6449 heartbeater.cc:511] Master 127.21.202.126:46849 requested a full tablet report, sending...
I20260806 19:46:30.467223 6262 ts_manager.cc:194] Registered new tserver with Master: d31ac125bf7d4fcbbb1065eca180cccf (127.21.202.65:41599)
I20260806 19:46:30.468880 6262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:55891
W20260806 19:46:30.849608 6453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:46:30.849958 6453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:46:30.850060 6453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:46:30.858283 6453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:46:30.858423 6453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:46:30.858623 6453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:46:30.870186 6453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:33417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=35831
--tserver_master_addrs=127.21.202.126:46849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:46:30.871836 6453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:46:30.873523 6453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:46:30.888257 6458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:30.888408 6459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:30.888157 6461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:46:30.889389 6453 server_base.cc:1034] running on GCE node
I20260806 19:46:30.890137 6453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:46:30.891242 6453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:46:30.892554 6453 hybrid_clock.cc:648] HybridClock initialized: now 1786045590892484 us; error 62 us; skew 500 ppm
I20260806 19:46:30.893036 6453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:46:30 dist-test-slave-77v9 krb5kdc[5673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045590, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:46:30.897075 6453 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:46:30.899245 6453 webserver.cc:460] Webserver started at http://127.21.202.66:35831/ using document root <none> and password file <none>
I20260806 19:46:30.900211 6453 fs_manager.cc:359] Metadata directory not provided
I20260806 19:46:30.900416 6453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:46:30.906811 6453 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 19:46:30.909966 6469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:30.911602 6453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 19:46:30.911926 6453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "061b183a9334432699e36f839ab4e4e3"
format_stamp: "Formatted at 2026-08-06 19:46:07 on dist-test-slave-77v9"
server_key: "4bf1a529ac8ac3839990035dff45d168"
server_key_iv: "dbf97b53c5e98187081acc34850e2015"
server_key_version: "encryptionkey@0"
I20260806 19:46:30.913214 6453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:46:30.930990 6453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:46:30.936275 6453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:46:30.936954 6453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:46:30.938186 6453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:46:30.939774 6453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:46:30.939940 6453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:30.940078 6453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:46:30.940173 6453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:30.984843 6453 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:33417
I20260806 19:46:30.984906 6582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:33417 every 8 connection(s)
I20260806 19:46:30.986406 6453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:46:30.991324 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 6453
I20260806 19:46:30.992132 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:39295
--local_ip_for_outbound_sockets=127.21.202.67
--tserver_master_addrs=127.21.202.126:46849
--webserver_port=38747
--webserver_interface=127.21.202.67
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:46:30 dist-test-slave-77v9 krb5kdc[5673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045590, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:46:31.012140 6585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:30.990084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:56599 (local address 127.21.202.126:46849)
0806 19:46:30.990614 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:30.990619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:30.991922 (+ 1303us) server_negotiation.cc:407] Connection header received
0806 19:46:30.995208 (+ 3286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:30.995218 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:30.995342 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:30.995476 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:30.997297 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:30.998455 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.000427 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.001197 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.001374 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.005581 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:31.005587 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:31.005591 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:31.005629 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:31.008539 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:31.009619 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:31.009624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:31.009626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:31.009752 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:31.010236 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:31.010241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:31.010243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:31.010532 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:31.010699 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:31.011477 (+ 778us) server_negotiation.cc:299] Negotiation successful
0806 19:46:31.011728 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":306,"threads_started":1}
I20260806 19:46:31.013412 6586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:30.990824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:56599)
0806 19:46:30.991712 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:30.991765 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:30.994825 (+ 3060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:30.995729 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:30.995737 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:30.996356 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:46:30.997084 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:30.997107 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:30.998687 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:30.998691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:31.000218 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:31.000232 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.001496 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.001501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:31.001719 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.002560 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:46:31.002585 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:46:31.005250 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:46:31.008723 (+ 3473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:46:31.008734 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:46:31.008757 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:46:31.009469 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:46:31.009886 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:46:31.009891 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:46:31.009895 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:46:31.010102 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:46:31.010673 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:46:31.010681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:46:31.011066 (+ 385us) client_negotiation.cc:769] Sending connection context
0806 19:46:31.011374 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 19:46:31.011945 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":375,"threads_started":1}
I20260806 19:46:31.015209 6583 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46849
I20260806 19:46:31.015620 6583 heartbeater.cc:464] Registering TS with master...
I20260806 19:46:31.016587 6583 heartbeater.cc:511] Master 127.21.202.126:46849 requested a full tablet report, sending...
I20260806 19:46:31.019160 6262 ts_manager.cc:194] Registered new tserver with Master: 061b183a9334432699e36f839ab4e4e3 (127.21.202.66:33417)
I20260806 19:46:31.020061 6262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:56599
W20260806 19:46:31.394264 6587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:46:31.394601 6587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:46:31.394825 6587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:46:31.402722 6587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:46:31.402868 6587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:46:31.403016 6587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:46:31.414161 6587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:39295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=38747
--tserver_master_addrs=127.21.202.126:46849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:46:31.415612 6587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:46:31.417281 6587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:46:31.436285 6592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:31.436745 6595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:46:31.437359 6587 server_base.cc:1034] running on GCE node
W20260806 19:46:31.437572 6593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:46:31.438441 6587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:46:31.440158 6587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:46:31.441423 6587 hybrid_clock.cc:648] HybridClock initialized: now 1786045591441315 us; error 72 us; skew 500 ppm
I20260806 19:46:31.441808 6587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:46:31 dist-test-slave-77v9 krb5kdc[5673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045591, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:46:31.446009 6587 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:46:31.447906 6587 webserver.cc:460] Webserver started at http://127.21.202.67:38747/ using document root <none> and password file <none>
I20260806 19:46:31.448668 6587 fs_manager.cc:359] Metadata directory not provided
I20260806 19:46:31.448796 6587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:46:31.457069 6587 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.006s
I20260806 19:46:31.460392 6602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:31.462173 6587 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:46:31.462409 6587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "6c0836279c1749fda03a4d41ad11588a"
format_stamp: "Formatted at 2026-08-06 19:46:08 on dist-test-slave-77v9"
server_key: "a83a9c314f97581dddf9b3cb18f2f9bd"
server_key_iv: "398ab31fbb63d3b65ca08011b047a288"
server_key_version: "encryptionkey@0"
I20260806 19:46:31.463779 6587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:46:31.474268 6449 heartbeater.cc:503] Master 127.21.202.126:46849 was elected leader, sending a full tablet report...
I20260806 19:46:31.492569 6587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:46:31.497874 6587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:46:31.498200 6587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:46:31.499362 6587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:46:31.501014 6587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:46:31.501142 6587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:31.501284 6587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:46:31.501376 6587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:31.543462 6587 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:39295
I20260806 19:46:31.543561 6715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:39295 every 8 connection(s)
I20260806 19:46:31.545100 6587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:46:31.551942 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 6587
Aug 06 19:46:31 dist-test-slave-77v9 krb5kdc[5673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045591, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:46:31.570176 6718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.548063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:47031 (local address 127.21.202.126:46849)
0806 19:46:31.548595 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:31.548600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:31.549519 (+ 919us) server_negotiation.cc:407] Connection header received
0806 19:46:31.551477 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:31.551482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:31.551592 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:31.551719 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:31.553799 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.555100 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.557235 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.557715 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.557880 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.562785 (+ 4905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:31.562791 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:31.562794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:31.562848 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:31.566349 (+ 3501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:31.567224 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:31.567229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:31.567233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:31.567379 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:31.567991 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:31.567997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:31.568000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:31.568396 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:31.568576 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:31.569524 (+ 948us) server_negotiation.cc:299] Negotiation successful
0806 19:46:31.569765 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":268,"threads_started":1}
I20260806 19:46:31.571287 6720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.553059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48780 (local address 127.21.202.126:46849)
0806 19:46:31.553650 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:31.553656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:31.554073 (+ 417us) server_negotiation.cc:407] Connection header received
0806 19:46:31.554243 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:31.554249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:31.554371 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:31.554497 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:31.556436 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.557407 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.559056 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.559710 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.559851 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.564346 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:31.564355 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:31.564358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:31.564403 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:31.567602 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:31.568694 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:31.568705 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:31.568709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:31.568894 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:31.569381 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:31.569386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:31.569390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:31.569681 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:31.569979 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:31.570652 (+ 673us) server_negotiation.cc:299] Negotiation successful
0806 19:46:31.570918 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":299,"threads_started":1}
I20260806 19:46:31.571436 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.548539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:47031)
0806 19:46:31.549329 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:31.549393 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:31.551192 (+ 1799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:31.551970 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:31.551979 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:31.552609 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:46:31.553599 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:31.553625 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.555288 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.555292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:31.556795 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:31.556861 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.557994 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.558005 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:31.558292 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.559177 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:46:31.559208 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:46:31.562473 (+ 3265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:46:31.566555 (+ 4082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:46:31.566568 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:46:31.566590 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:46:31.567078 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:46:31.567506 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:46:31.567512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:46:31.567515 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:46:31.567751 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:46:31.568551 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:46:31.568559 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:46:31.569089 (+ 530us) client_negotiation.cc:769] Sending connection context
0806 19:46:31.569412 (+ 323us) client_negotiation.cc:240] Negotiation successful
0806 19:46:31.570011 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":345,"threads_started":1}
I20260806 19:46:31.573685 6716 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46849
I20260806 19:46:31.574221 6716 heartbeater.cc:464] Registering TS with master...
I20260806 19:46:31.575053 6716 heartbeater.cc:511] Master 127.21.202.126:46849 requested a full tablet report, sending...
I20260806 19:46:31.576632 6262 ts_manager.cc:194] Registered new tserver with Master: 6c0836279c1749fda03a4d41ad11588a (127.21.202.67:39295)
I20260806 19:46:31.577576 6262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:47031
I20260806 19:46:31.587136 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:46:31.595906 6720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.588216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48796 (local address 127.21.202.126:46849)
0806 19:46:31.588779 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:31.588786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:31.589587 (+ 801us) server_negotiation.cc:407] Connection header received
0806 19:46:31.589902 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:31.589908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:31.590096 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:31.590279 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:46:31.590902 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.591661 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.593457 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.594115 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.594282 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.594856 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:46:31.595164 (+ 308us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 19:46:31.595577 (+ 413us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48796: token expired
Metrics: {"server-negotiator.queue_time_us":83}
W20260806 19:46:31.596036 6720 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48796: token expired
W20260806 19:46:31.596764 6183 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.21.202.126:46849)
I20260806 19:46:31.611698 6720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.597458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.21.202.126:46849)
0806 19:46:31.597770 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:31.597778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:31.597798 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:46:31.598018 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:31.598022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:31.598091 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:31.598198 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:31.599707 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.600446 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.601940 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.602366 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.602508 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.605518 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:31.605524 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:31.605527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:31.605565 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:31.608718 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:31.609512 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:31.609517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:31.609521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:31.609669 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:31.610082 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:31.610088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:31.610091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:31.610491 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:31.610753 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:31.611120 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 19:46:31.611367 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260806 19:46:31.615638 6262 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:46:31.623541 6262 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:46:31.666039 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.638927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56952 (local address 127.21.202.66:33417)
0806 19:46:31.654859 (+ 15932us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:31.654870 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:31.654902 (+ 32us) server_negotiation.cc:407] Connection header received
0806 19:46:31.655011 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:31.655019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:31.655325 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:31.655549 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:46:31.657224 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.659372 (+ 2148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.662866 (+ 3494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.664415 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.664567 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.664836 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:31.665451 (+ 615us) server_negotiation.cc:299] Negotiation successful
0806 19:46:31.665650 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15726,"thread_start_us":15637,"threads_started":1}
I20260806 19:46:31.666092 6727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.638596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:33417 (local address 127.0.0.1:56952)
0806 19:46:31.653908 (+ 15312us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:31.653981 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:31.654229 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:31.656127 (+ 1898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:31.656139 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:31.656187 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:46:31.656856 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:31.656983 (+ 127us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.659600 (+ 2617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.659610 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:31.662651 (+ 3041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:31.662668 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.664852 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.664862 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:31.665086 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.665205 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 19:46:31.665362 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 19:46:31.665576 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14900,"mutex_wait_us":34,"spinlock_wait_cycles":41344,"thread_start_us":15891,"threads_started":1}
I20260806 19:46:31.668787 6730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.640049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44348 (local address 127.21.202.65:41599)
0806 19:46:31.658346 (+ 18297us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:31.658358 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:31.658391 (+ 33us) server_negotiation.cc:407] Connection header received
0806 19:46:31.658501 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:31.658509 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:31.658806 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:31.659071 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:46:31.660554 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.662126 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.664988 (+ 2862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.666993 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.667132 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.667373 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:31.668227 (+ 854us) server_negotiation.cc:299] Negotiation successful
0806 19:46:31.668416 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18088,"thread_start_us":18039,"threads_started":1}
I20260806 19:46:31.669155 6729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.639330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:41599 (local address 127.0.0.1:44348)
0806 19:46:31.656039 (+ 16709us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:31.656078 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:31.656509 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:31.659393 (+ 2884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:31.659399 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:31.659437 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:46:31.660152 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:31.660167 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.662279 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.662290 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:31.664770 (+ 2480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:31.664786 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.667887 (+ 3101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.667894 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:31.668019 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.668045 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 19:46:31.668167 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 19:46:31.668382 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16445,"spinlock_wait_cycles":1408,"thread_start_us":16487,"threads_started":1}
I20260806 19:46:31.669514 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.658510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:39295 (local address 127.0.0.1:56006)
0806 19:46:31.660728 (+ 2218us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:31.660747 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:31.660889 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:31.661524 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:31.661530 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:31.661557 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:46:31.661923 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:31.661934 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.664455 (+ 2521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.664463 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:31.666799 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:31.666811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.668681 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.668688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:31.668901 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.668937 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 19:46:31.669036 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 19:46:31.669186 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2048,"thread_start_us":1873,"threads_started":1}
I20260806 19:46:31.669799 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.655796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56006 (local address 127.21.202.67:39295)
0806 19:46:31.660440 (+ 4644us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:31.660448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:31.660937 (+ 489us) server_negotiation.cc:407] Connection header received
0806 19:46:31.661050 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:31.661059 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:31.661305 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:31.661504 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:46:31.662375 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.664231 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.666989 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.668144 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.668283 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.668512 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:31.669175 (+ 663us) server_negotiation.cc:299] Negotiation successful
0806 19:46:31.669380 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4413,"thread_start_us":4194,"threads_started":1}
I20260806 19:46:31.671238 6517 tablet_service.cc:1467] Processing CreateTablet for tablet 3e69c5115702483ca438de9872f8fbda (DEFAULT_TABLE table=test-table [id=c9b4347df8a44b49888b190b0895c0bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:46:31.672909 6517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e69c5115702483ca438de9872f8fbda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:46:31.673707 6650 tablet_service.cc:1467] Processing CreateTablet for tablet 3e69c5115702483ca438de9872f8fbda (DEFAULT_TABLE table=test-table [id=c9b4347df8a44b49888b190b0895c0bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:46:31.673836 6383 tablet_service.cc:1467] Processing CreateTablet for tablet 3e69c5115702483ca438de9872f8fbda (DEFAULT_TABLE table=test-table [id=c9b4347df8a44b49888b190b0895c0bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:46:31.674975 6650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e69c5115702483ca438de9872f8fbda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:46:31.675323 6383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e69c5115702483ca438de9872f8fbda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:46:31.689112 6733 tablet_bootstrap.cc:492] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a: Bootstrap starting.
I20260806 19:46:31.691030 6734 tablet_bootstrap.cc:492] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3: Bootstrap starting.
I20260806 19:46:31.693532 6733 tablet_bootstrap.cc:654] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a: Neither blocks nor log segments found. Creating new log.
I20260806 19:46:31.695122 6733 log.cc:824] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:46:31.695724 6734 tablet_bootstrap.cc:654] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3: Neither blocks nor log segments found. Creating new log.
I20260806 19:46:31.697126 6735 tablet_bootstrap.cc:492] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf: Bootstrap starting.
I20260806 19:46:31.698593 6734 log.cc:824] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3: Log is configured to *not* fsync() on all Append() calls
I20260806 19:46:31.700546 6733 tablet_bootstrap.cc:492] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a: No bootstrap required, opened a new log
I20260806 19:46:31.700963 6733 ts_tablet_manager.cc:1397] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.006s
I20260806 19:46:31.702219 6734 tablet_bootstrap.cc:492] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3: No bootstrap required, opened a new log
I20260806 19:46:31.702378 6735 tablet_bootstrap.cc:654] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf: Neither blocks nor log segments found. Creating new log.
I20260806 19:46:31.702553 6734 ts_tablet_manager.cc:1397] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260806 19:46:31.703886 6735 log.cc:824] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf: Log is configured to *not* fsync() on all Append() calls
I20260806 19:46:31.707973 6735 tablet_bootstrap.cc:492] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf: No bootstrap required, opened a new log
I20260806 19:46:31.708303 6735 ts_tablet_manager.cc:1397] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260806 19:46:31.711031 6733 raft_consensus.cc:348] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:31.711517 6733 raft_consensus.cc:374] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:46:31.711637 6733 raft_consensus.cc:729] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c0836279c1749fda03a4d41ad11588a, State: Initialized, Role: FOLLOWER
I20260806 19:46:31.712401 6733 consensus_queue.cc:260] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:31.713830 6716 heartbeater.cc:503] Master 127.21.202.126:46849 was elected leader, sending a full tablet report...
I20260806 19:46:31.714496 6734 raft_consensus.cc:348] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:31.714893 6733 ts_tablet_manager.cc:1428] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260806 19:46:31.715013 6734 raft_consensus.cc:374] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:46:31.715109 6734 raft_consensus.cc:729] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 061b183a9334432699e36f839ab4e4e3, State: Initialized, Role: FOLLOWER
I20260806 19:46:31.715837 6734 consensus_queue.cc:260] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:31.717764 6583 heartbeater.cc:503] Master 127.21.202.126:46849 was elected leader, sending a full tablet report...
I20260806 19:46:31.718025 6734 ts_tablet_manager.cc:1428] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260806 19:46:31.722005 6735 raft_consensus.cc:348] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:31.722625 6735 raft_consensus.cc:374] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:46:31.722759 6735 raft_consensus.cc:729] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d31ac125bf7d4fcbbb1065eca180cccf, State: Initialized, Role: FOLLOWER
I20260806 19:46:31.723619 6735 consensus_queue.cc:260] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:31.725454 6735 ts_tablet_manager.cc:1428] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf: Time spent starting tablet: real 0.017s user 0.013s sys 0.005s
W20260806 19:46:31.739233 6584 tablet.cc:2369] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:46:31.796960 6717 tablet.cc:2369] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:46:31.931953 6450 tablet.cc:2369] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:46:31.986827 6740 raft_consensus.cc:482] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:46:31.987226 6740 raft_consensus.cc:504] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:31.990356 6740 leader_election.cc:288] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c0836279c1749fda03a4d41ad11588a (127.21.202.67:39295), d31ac125bf7d4fcbbb1065eca180cccf (127.21.202.65:41599)
I20260806 19:46:32.000640 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.991178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:41599 (local address 127.21.202.66:50959)
0806 19:46:31.993545 (+ 2367us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:31.993568 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:31.993752 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:31.994144 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:31.994149 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:31.994250 (+ 101us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:46:31.994772 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:31.994785 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.996176 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.996183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:31.998211 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:31.998228 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.999740 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.999749 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:31.999928 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.999957 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 19:46:32.000039 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 19:46:32.000147 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2231,"mutex_wait_us":4,"spinlock_wait_cycles":26496,"thread_start_us":1814,"threads_started":1}
I20260806 19:46:32.000706 6730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.991330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:50959 (local address 127.21.202.65:41599)
0806 19:46:31.991616 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:31.991623 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:31.993655 (+ 2032us) server_negotiation.cc:407] Connection header received
0806 19:46:31.993911 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:31.993918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:31.993998 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:31.994092 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:46:31.994964 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.996062 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.998409 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.999474 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.999645 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:31.999728 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:32.000184 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 19:46:32.000349 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260806 19:46:32.001323 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.991680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:48305 (local address 127.21.202.67:39295)
0806 19:46:31.991947 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:31.991955 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:31.993373 (+ 1418us) server_negotiation.cc:407] Connection header received
0806 19:46:31.993735 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:31.993742 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:31.993833 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:31.993941 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:46:31.994717 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.995951 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.998970 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:32.000204 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:32.000367 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:32.000513 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:32.000952 (+ 439us) server_negotiation.cc:299] Negotiation successful
0806 19:46:32.001103 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 19:46:32.002107 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:31.991052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:39295 (local address 127.21.202.66:48305)
0806 19:46:31.992991 (+ 1939us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:31.993017 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:46:31.993522 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:46:31.993950 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:46:31.993955 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:46:31.993978 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:46:31.994557 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:31.994569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:31.996108 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:31.996115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:31.998774 (+ 2659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:46:31.998790 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:46:32.000449 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:46:32.000461 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:46:32.000688 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:32.000730 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 19:46:32.001530 (+ 800us) client_negotiation.cc:240] Negotiation successful
0806 19:46:32.001690 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1670,"mutex_wait_us":48,"thread_start_us":1171,"threads_started":1}
I20260806 19:46:32.001861 6404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e69c5115702483ca438de9872f8fbda" candidate_uuid: "061b183a9334432699e36f839ab4e4e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" is_pre_election: true
I20260806 19:46:32.002516 6404 raft_consensus.cc:2454] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 061b183a9334432699e36f839ab4e4e3 in term 0.
I20260806 19:46:32.003209 6670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e69c5115702483ca438de9872f8fbda" candidate_uuid: "061b183a9334432699e36f839ab4e4e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c0836279c1749fda03a4d41ad11588a" is_pre_election: true
I20260806 19:46:32.003291 6473 leader_election.cc:302] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 061b183a9334432699e36f839ab4e4e3, d31ac125bf7d4fcbbb1065eca180cccf; no voters:
I20260806 19:46:32.003705 6670 raft_consensus.cc:2454] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 061b183a9334432699e36f839ab4e4e3 in term 0.
I20260806 19:46:32.003758 6740 raft_consensus.cc:2781] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:46:32.003916 6740 raft_consensus.cc:482] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:46:32.004033 6740 raft_consensus.cc:3037] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:46:32.006521 6740 raft_consensus.cc:504] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:32.007194 6740 leader_election.cc:288] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [CANDIDATE]: Term 1 election: Requested vote from peers 6c0836279c1749fda03a4d41ad11588a (127.21.202.67:39295), d31ac125bf7d4fcbbb1065eca180cccf (127.21.202.65:41599)
I20260806 19:46:32.007822 6670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e69c5115702483ca438de9872f8fbda" candidate_uuid: "061b183a9334432699e36f839ab4e4e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c0836279c1749fda03a4d41ad11588a"
I20260806 19:46:32.007815 6404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e69c5115702483ca438de9872f8fbda" candidate_uuid: "061b183a9334432699e36f839ab4e4e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d31ac125bf7d4fcbbb1065eca180cccf"
I20260806 19:46:32.008052 6670 raft_consensus.cc:3037] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:46:32.008054 6404 raft_consensus.cc:3037] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:46:32.011479 6404 raft_consensus.cc:2454] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 061b183a9334432699e36f839ab4e4e3 in term 1.
I20260806 19:46:32.011479 6670 raft_consensus.cc:2454] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 061b183a9334432699e36f839ab4e4e3 in term 1.
I20260806 19:46:32.012056 6473 leader_election.cc:302] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 061b183a9334432699e36f839ab4e4e3, 6c0836279c1749fda03a4d41ad11588a, d31ac125bf7d4fcbbb1065eca180cccf; no voters:
I20260806 19:46:32.012540 6740 raft_consensus.cc:2781] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:46:32.013295 6740 raft_consensus.cc:686] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [term 1 LEADER]: Becoming Leader. State: Replica: 061b183a9334432699e36f839ab4e4e3, State: Running, Role: LEADER
I20260806 19:46:32.013950 6740 consensus_queue.cc:237] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } }
I20260806 19:46:32.019649 6261 catalog_manager.cc:5495] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 061b183a9334432699e36f839ab4e4e3 (127.21.202.66). New cstate: current_term: 1 leader_uuid: "061b183a9334432699e36f839ab4e4e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061b183a9334432699e36f839ab4e4e3" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 } health_report { overall_health: UNKNOWN } } }
I20260806 19:46:32.049001 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:32.041286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56956 (local address 127.21.202.66:33417)
0806 19:46:32.041604 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:32.041611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:32.041638 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:46:32.041783 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:32.041788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:32.041914 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:32.042259 (+ 345us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:46:32.043013 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:32.043818 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:32.045689 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:32.046244 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:32.046377 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:32.047374 (+ 997us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:46:32.048057 (+ 683us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:46:32.048233 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:32.048545 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 19:46:32.048635 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 19:46:32.061002 6670 raft_consensus.cc:1261] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a [term 1 FOLLOWER]: Refusing update from remote peer 061b183a9334432699e36f839ab4e4e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:46:32.061007 6404 raft_consensus.cc:1261] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf [term 1 FOLLOWER]: Refusing update from remote peer 061b183a9334432699e36f839ab4e4e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:46:32.062486 6740 consensus_queue.cc:1035] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c0836279c1749fda03a4d41ad11588a" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 39295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:46:32.062947 6744 consensus_queue.cc:1035] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d31ac125bf7d4fcbbb1065eca180cccf" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 41599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:46:32.081483 6748 mvcc.cc:205] Tried to move back new op lower bound from 7315642745067089920 to 7315642744893632512. Current Snapshot: MvccSnapshot[applied={T|T < 7315642745067089920}]
I20260806 19:46:32.100919 6262 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:46:32.101245 6262 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:46:32.105525 6262 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Sending DeleteTablet for 3 replicas of tablet 3e69c5115702483ca438de9872f8fbda
I20260806 19:46:32.107075 6650 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e69c5115702483ca438de9872f8fbda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:46:32 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:56006
I20260806 19:46:32.107460 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 6299
I20260806 19:46:32.107328 6517 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e69c5115702483ca438de9872f8fbda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:46:32 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:56952
I20260806 19:46:32.107903 6383 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e69c5115702483ca438de9872f8fbda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:46:32 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:44348
I20260806 19:46:32.108480 6756 tablet_replica.cc:330] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a: stopping tablet replica
I20260806 19:46:32.109023 6756 raft_consensus.cc:2229] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:46:32.109609 6756 raft_consensus.cc:2258] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:46:32.112371 6756 ts_tablet_manager.cc:1905] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:46:32.122329 6756 ts_tablet_manager.cc:1918] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:46:32.122581 6756 log.cc:1196] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/3e69c5115702483ca438de9872f8fbda
I20260806 19:46:32.123647 6756 ts_tablet_manager.cc:1939] T 3e69c5115702483ca438de9872f8fbda P 6c0836279c1749fda03a4d41ad11588a: Deleting consensus metadata
I20260806 19:46:32.126295 6246 catalog_manager.cc:4841] TS 6c0836279c1749fda03a4d41ad11588a (127.21.202.67:39295): tablet 3e69c5115702483ca438de9872f8fbda (table test-table [id=c9b4347df8a44b49888b190b0895c0bf]) successfully deleted
I20260806 19:46:32.130369 6758 tablet_replica.cc:330] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf: stopping tablet replica
I20260806 19:46:32.130995 6758 raft_consensus.cc:2229] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:46:32.131620 6758 raft_consensus.cc:2258] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:46:32.132555 6757 tablet_replica.cc:330] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3: stopping tablet replica
I20260806 19:46:32.133740 6757 raft_consensus.cc:2229] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:46:32.134502 6757 raft_consensus.cc:2258] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:46:32.135471 6758 ts_tablet_manager.cc:1905] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:46:32.136749 6757 ts_tablet_manager.cc:1905] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:46:32.145381 6757 ts_tablet_manager.cc:1918] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:46:32.145629 6757 log.cc:1196] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/3e69c5115702483ca438de9872f8fbda
I20260806 19:46:32.146418 6757 ts_tablet_manager.cc:1939] T 3e69c5115702483ca438de9872f8fbda P 061b183a9334432699e36f839ab4e4e3: Deleting consensus metadata
I20260806 19:46:32.147346 6758 ts_tablet_manager.cc:1918] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:46:32.147599 6758 log.cc:1196] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/3e69c5115702483ca438de9872f8fbda
I20260806 19:46:32.148248 6758 ts_tablet_manager.cc:1939] T 3e69c5115702483ca438de9872f8fbda P d31ac125bf7d4fcbbb1065eca180cccf: Deleting consensus metadata
I20260806 19:46:32.148993 6246 catalog_manager.cc:4841] TS 061b183a9334432699e36f839ab4e4e3 (127.21.202.66:33417): tablet 3e69c5115702483ca438de9872f8fbda (table test-table [id=c9b4347df8a44b49888b190b0895c0bf]) successfully deleted
I20260806 19:46:32.150329 6249 catalog_manager.cc:4841] TS d31ac125bf7d4fcbbb1065eca180cccf (127.21.202.65:41599): tablet 3e69c5115702483ca438de9872f8fbda (table test-table [id=c9b4347df8a44b49888b190b0895c0bf]) successfully deleted
I20260806 19:46:32.156183 6730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:32.109020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44358 (local address 127.21.202.65:41599)
0806 19:46:32.109218 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:32.109224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:32.129569 (+ 20345us) server_negotiation.cc:407] Connection header received
0806 19:46:32.129830 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:32.129838 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:32.129967 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:32.130095 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:32.132429 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:32.133950 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:32.143715 (+ 9765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:32.144404 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:32.144560 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:32.149166 (+ 4606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:32.149181 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:32.149200 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:32.149254 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:32.152657 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:32.153647 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:32.153655 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:32.153657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:32.153807 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:32.154316 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:32.154321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:32.154323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:32.154730 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:32.155065 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:32.155443 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 19:46:32.155718 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 19:46:32.157084 6444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:46:32.256034 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 6299
I20260806 19:46:32.281906 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 6453
I20260806 19:46:32.298300 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:32.282837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56968 (local address 127.21.202.66:33417)
0806 19:46:32.283051 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:32.283058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:32.283082 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:46:32.283255 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:32.283260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:32.283347 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:32.283447 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:32.284888 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:32.285771 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:32.287114 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:32.287576 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:32.287719 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:32.290715 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:32.290732 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:32.290748 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:32.290800 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:32.294835 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:32.295698 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:32.295710 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:32.295719 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:32.295865 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:32.296479 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:32.296485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:32.296489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:32.296957 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:32.297336 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:32.297645 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 19:46:32.297891 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 19:46:32.299060 6578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:46:32.419750 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 6453
I20260806 19:46:32.442495 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 6587
I20260806 19:46:32.458572 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:32.443443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56018 (local address 127.21.202.67:39295)
0806 19:46:32.443661 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 19:46:32.443667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:46:32.443690 (+ 23us) server_negotiation.cc:407] Connection header received
0806 19:46:32.443839 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:46:32.443843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:46:32.443936 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:46:32.444054 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:46:32.445541 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:32.446285 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:32.447628 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:46:32.448064 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:46:32.448213 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:46:32.451116 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:46:32.451132 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:46:32.451150 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:46:32.451192 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:46:32.454538 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:32.455412 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:32.455420 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:32.455422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:32.455584 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:46:32.456332 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:46:32.456336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:46:32.456339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:46:32.456771 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:46:32.457378 (+ 607us) server_negotiation.cc:1035] Waiting for connection context
0806 19:46:32.457630 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 19:46:32.458002 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 19:46:32.459532 6711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:46:32.557129 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 6587
I20260806 19:46:32.577319 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 6229
I20260806 19:46:32.578600 6292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:46:32.709540 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 6229
I20260806 19:46:52.737521 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46849
--webserver_interface=127.21.202.126
--webserver_port=44555
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:46849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:46:53.125186 6764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:46:53.125577 6764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:46:53.125739 6764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:46:53.133777 6764 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 19:46:53.133947 6764 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 19:46:53.134042 6764 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 19:46:53.134138 6764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:46:53.134241 6764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:46:53.134367 6764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:46:53.134460 6764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:46:53.134549 6764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:46:53.145200 6764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:46849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=44555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:46:53.146673 6764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:46:53.148466 6764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:46:53.162712 6770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:53.162778 6772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:53.162746 6769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:46:53.163597 6764 server_base.cc:1034] running on GCE node
I20260806 19:46:53.164561 6764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:46:53.166203 6764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:46:53.167448 6764 hybrid_clock.cc:648] HybridClock initialized: now 1786045613167381 us; error 39 us; skew 500 ppm
I20260806 19:46:53.167835 6764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:46:53 dist-test-slave-77v9 krb5kdc[5673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045613, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:46:53.175518 6764 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:46:53.177302 6764 webserver.cc:460] Webserver started at http://127.21.202.126:44555/ using document root <none> and password file <none>
I20260806 19:46:53.178089 6764 fs_manager.cc:359] Metadata directory not provided
I20260806 19:46:53.178228 6764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:46:53.185086 6764 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 19:46:53.188562 6779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:53.190173 6764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:46:53.190397 6764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "7d4dc71690334648b218d29a91b1335a"
format_stamp: "Formatted at 2026-08-06 19:46:05 on dist-test-slave-77v9"
server_key: "8c892914178612f5253e810bc6aa7086"
server_key_iv: "20462370340cc7784d3c98aab4fa70e8"
server_key_version: "encryptionkey@0"
I20260806 19:46:53.191542 6764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:46:53.205332 6764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:46:53.210237 6764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:46:53.210568 6764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:46:53.231659 6764 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:46849
I20260806 19:46:53.231701 6831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:46849 every 8 connection(s)
I20260806 19:46:53.233304 6764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:46:53.234794 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 6764
I20260806 19:46:53.235924 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:41599
--local_ip_for_outbound_sockets=127.21.202.65
--tserver_master_addrs=127.21.202.126:46849
--webserver_port=37395
--webserver_interface=127.21.202.65
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:46:53.243978 6832 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:46:53.249260 6832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Bootstrap starting.
I20260806 19:46:53.270531 6832 log.cc:824] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:46:53.288074 6832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:46:53.288746 6832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Bootstrap complete.
I20260806 19:46:53.301182 6832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } }
I20260806 19:46:53.302232 6832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d4dc71690334648b218d29a91b1335a, State: Initialized, Role: FOLLOWER
I20260806 19:46:53.303097 6832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } }
I20260806 19:46:53.303444 6832 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:46:53.303634 6832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:46:53.303851 6832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 2 FOLLOWER]: Advancing to term 3
I20260806 19:46:53.308969 6832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } }
I20260806 19:46:53.309840 6832 leader_election.cc:302] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d4dc71690334648b218d29a91b1335a; no voters:
I20260806 19:46:53.310734 6832 leader_election.cc:288] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 19:46:53.310897 6836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 3 FOLLOWER]: Leader election won for term 3
I20260806 19:46:53.312366 6836 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [term 3 LEADER]: Becoming Leader. State: Replica: 7d4dc71690334648b218d29a91b1335a, State: Running, Role: LEADER
I20260806 19:46:53.313050 6836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } }
I20260806 19:46:53.313248 6832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:46:53.315712 6838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d4dc71690334648b218d29a91b1335a. Latest consensus state: current_term: 3 leader_uuid: "7d4dc71690334648b218d29a91b1335a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } } }
I20260806 19:46:53.316519 6838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: This master's current role is: LEADER
I20260806 19:46:53.316174 6837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7d4dc71690334648b218d29a91b1335a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4dc71690334648b218d29a91b1335a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46849 } } }
I20260806 19:46:53.317138 6837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a [sys.catalog]: This master's current role is: LEADER
I20260806 19:46:53.319845 6843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:46:53.330360 6843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:46:53.332015 6843 catalog_manager.cc:1259] Loaded cluster ID: e038041afe914edc8f292fb2336fcf0a
I20260806 19:46:53.332190 6843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:46:53.336436 6843 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:46:53.338620 6843 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Loaded TSK: 0
I20260806 19:46:53.338908 6843 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7d4dc71690334648b218d29a91b1335a: Loaded TSK: 1
F20260806 19:46:53.340173 6843 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb551a519ad google::LogMessage::Fail() at ??:0
@ 0x7fb551a5488a google::LogMessage::SendToLog() at ??:0
@ 0x7fb551a514de google::LogMessage::Flush() at ??:0
@ 0x7fb551a52a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb5557b57d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb5557a473e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb5674801c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fb56747f783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb56750b2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fb56750b0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fb5675555e9 std::function<>::operator()() at ??:0
@ 0x7fb567483cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fb567481c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb567509610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fb567509439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb55469d345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb5546a091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fb5546a0799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb55467beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb5579556db start_thread at ??:0
@ 0x7fb5508eb61f clone at ??:0
W20260806 19:46:53.703703 6834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:46:53.704031 6834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:46:53.704131 6834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:46:53.711896 6834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:46:53.712034 6834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:46:53.712155 6834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:46:53.724282 6834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:41599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=37395
--tserver_master_addrs=127.21.202.126:46849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:46:53.725989 6834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:46:53.727806 6834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:46:53.744903 6879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:53.744956 6882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:53.745030 6880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:46:53.745688 6834 server_base.cc:1034] running on GCE node
I20260806 19:46:53.746510 6834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:46:53.747610 6834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:46:53.748931 6834 hybrid_clock.cc:648] HybridClock initialized: now 1786045613748874 us; error 35 us; skew 500 ppm
I20260806 19:46:53.749313 6834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:46:53 dist-test-slave-77v9 krb5kdc[5673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045613, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:46:53.752964 6834 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:46:53.754655 6834 webserver.cc:460] Webserver started at http://127.21.202.65:37395/ using document root <none> and password file <none>
I20260806 19:46:53.755424 6834 fs_manager.cc:359] Metadata directory not provided
I20260806 19:46:53.755582 6834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:46:53.762028 6834 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 19:46:53.765494 6889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:53.767414 6834 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 19:46:53.767637 6834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "d31ac125bf7d4fcbbb1065eca180cccf"
format_stamp: "Formatted at 2026-08-06 19:46:06 on dist-test-slave-77v9"
server_key: "de83fef3d9da6c352c0205b32a953806"
server_key_iv: "a998c1e1338789ab540e95df9c29d866"
server_key_version: "encryptionkey@0"
I20260806 19:46:53.768769 6834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:46:53.790586 6834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:46:53.796005 6834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:46:53.796324 6834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:46:53.797552 6834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:46:53.799054 6834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:46:53.799201 6834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:53.799336 6834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:46:53.799428 6834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:53.840643 6834 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:41599
I20260806 19:46:53.840749 7002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:41599 every 8 connection(s)
I20260806 19:46:53.842278 6834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:46:53.843889 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 6834
I20260806 19:46:53.844743 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:33417
--local_ip_for_outbound_sockets=127.21.202.66
--tserver_master_addrs=127.21.202.126:46849
--webserver_port=35831
--webserver_interface=127.21.202.66
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:46:53.848488 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:53.845963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:33471)
0806 19:46:53.846889 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:53.847236 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":337,"threads_started":1}
W20260806 19:46:53.849391 7003 heartbeater.cc:644] Failed to heartbeat to 127.21.202.126:46849 (0 consecutive failures): Network error: Failed to ping master at 127.21.202.126:46849: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
I20260806 19:46:53.850560 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:53.850078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:47081)
0806 19:46:53.850290 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:53.850394 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 19:46:53.851601 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:53.851223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:39377)
0806 19:46:53.851389 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:53.851480 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 19:46:53.851922 7003 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 19:46:54.213413 7006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:46:54.213779 7006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:46:54.213976 7006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:46:54.221839 7006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:46:54.221980 7006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:46:54.222173 7006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:46:54.233680 7006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:33417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=35831
--tserver_master_addrs=127.21.202.126:46849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:46:54.235181 7006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:46:54.237257 7006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:46:54.252465 7011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:54.252419 7012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:54.252503 7014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:46:54.253791 7006 server_base.cc:1034] running on GCE node
I20260806 19:46:54.254525 7006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:46:54.255587 7006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:46:54.256852 7006 hybrid_clock.cc:648] HybridClock initialized: now 1786045614256734 us; error 62 us; skew 500 ppm
I20260806 19:46:54.257233 7006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:46:54 dist-test-slave-77v9 krb5kdc[5673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045614, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:46:54.260988 7006 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:46:54.262660 7006 webserver.cc:460] Webserver started at http://127.21.202.66:35831/ using document root <none> and password file <none>
I20260806 19:46:54.263422 7006 fs_manager.cc:359] Metadata directory not provided
I20260806 19:46:54.263654 7006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:46:54.269948 7006 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260806 19:46:54.273216 7021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:54.274763 7006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 19:46:54.274964 7006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "061b183a9334432699e36f839ab4e4e3"
format_stamp: "Formatted at 2026-08-06 19:46:07 on dist-test-slave-77v9"
server_key: "4bf1a529ac8ac3839990035dff45d168"
server_key_iv: "dbf97b53c5e98187081acc34850e2015"
server_key_version: "encryptionkey@0"
I20260806 19:46:54.276085 7006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:46:54.300165 7006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:46:54.305563 7006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:46:54.305898 7006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:46:54.307111 7006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:46:54.308681 7006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:46:54.308818 7006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:54.308996 7006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:46:54.309095 7006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:54.347193 7006 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:33417
I20260806 19:46:54.347301 7133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:33417 every 8 connection(s)
I20260806 19:46:54.348883 7006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:46:54.352945 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 7006
I20260806 19:46:54.353597 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:39295
--local_ip_for_outbound_sockets=127.21.202.67
--tserver_master_addrs=127.21.202.126:46849
--webserver_port=38747
--webserver_interface=127.21.202.67
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:46:54.354384 7136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:54.352218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:46621)
0806 19:46:54.353111 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:54.353408 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":359,"threads_started":1}
W20260806 19:46:54.355082 7134 heartbeater.cc:644] Failed to heartbeat to 127.21.202.126:46849 (0 consecutive failures): Network error: Failed to ping master at 127.21.202.126:46849: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
I20260806 19:46:54.355947 7136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:54.355612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:35797)
0806 19:46:54.355765 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:54.355845 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 19:46:54.356762 7136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:54.356435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:46015)
0806 19:46:54.356576 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:54.356655 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 19:46:54.357050 7134 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 19:46:54.709260 7137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:46:54.709595 7137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:46:54.709707 7137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:46:54.718158 7137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:46:54.718294 7137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:46:54.718423 7137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:46:54.728701 7137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:39295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=38747
--tserver_master_addrs=127.21.202.126:46849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:46:54.730494 7137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:46:54.731886 7137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:46:54.747556 7142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:54.747705 7145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:46:54.748064 7143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:46:54.748586 7137 server_base.cc:1034] running on GCE node
I20260806 19:46:54.749356 7137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:46:54.750389 7137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:46:54.751650 7137 hybrid_clock.cc:648] HybridClock initialized: now 1786045614751590 us; error 69 us; skew 500 ppm
I20260806 19:46:54.752017 7137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:46:54 dist-test-slave-77v9 krb5kdc[5673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045614, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:46:54.756090 7137 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:46:54.758798 7137 webserver.cc:460] Webserver started at http://127.21.202.67:38747/ using document root <none> and password file <none>
I20260806 19:46:54.759716 7137 fs_manager.cc:359] Metadata directory not provided
I20260806 19:46:54.759897 7137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:46:54.766412 7137 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 19:46:54.769956 7152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:54.771595 7137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:46:54.771814 7137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "6c0836279c1749fda03a4d41ad11588a"
format_stamp: "Formatted at 2026-08-06 19:46:08 on dist-test-slave-77v9"
server_key: "a83a9c314f97581dddf9b3cb18f2f9bd"
server_key_iv: "398ab31fbb63d3b65ca08011b047a288"
server_key_version: "encryptionkey@0"
I20260806 19:46:54.773221 7137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:46:54.799898 7137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:46:54.806058 7137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:46:54.806416 7137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:46:54.807588 7137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:46:54.809198 7137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:46:54.809341 7137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:54.809469 7137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:46:54.809559 7137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:46:54.847976 7137 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:39295
I20260806 19:46:54.848150 7264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:39295 every 8 connection(s)
I20260806 19:46:54.849542 7137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:46:54.853832 7267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:54.852509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:43157)
0806 19:46:54.853351 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:54.853516 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":524,"threads_started":1}
I20260806 19:46:54.855072 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:54.852909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:56543)
0806 19:46:54.853874 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:54.854203 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":378,"threads_started":1}
W20260806 19:46:54.855787 7265 heartbeater.cc:644] Failed to heartbeat to 127.21.202.126:46849 (0 consecutive failures): Network error: Failed to ping master at 127.21.202.126:46849: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
I20260806 19:46:54.856758 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:54.856393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:37855)
0806 19:46:54.856570 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:54.856648 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 19:46:54.858260 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 7137
I20260806 19:46:54.858449 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:54.857900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:47107)
0806 19:46:54.858140 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:54.858265 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260806 19:46:54.858844 7265 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 19:46:55.358563 7270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:55.357690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:56633)
0806 19:46:55.358188 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:55.358313 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":248,"threads_started":1}
I20260806 19:46:55.855670 7271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:55.854895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:43467)
0806 19:46:55.855403 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:55.855502 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":243,"threads_started":1}
I20260806 19:46:55.860299 7272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:55.859440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:60725)
0806 19:46:55.859965 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:55.860058 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":262,"threads_started":1}
I20260806 19:46:56.360535 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:56.359650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:47889)
0806 19:46:56.360258 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:56.360361 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":341,"threads_started":1}
I20260806 19:46:56.857549 7274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:56.856759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:44929)
0806 19:46:56.857274 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:56.857376 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":284,"threads_started":1}
I20260806 19:46:56.862285 7275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:56.861378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:44119)
0806 19:46:56.862009 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:56.862102 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":265,"threads_started":1}
I20260806 19:46:57.362818 7276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:57.361839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:49235)
0806 19:46:57.362473 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:57.362609 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":381,"threads_started":1}
I20260806 19:46:57.859609 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:57.858763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:54587)
0806 19:46:57.859341 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:57.859429 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":309,"threads_started":1}
I20260806 19:46:57.864166 7278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:57.863357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:41725)
0806 19:46:57.863840 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:57.863956 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":231,"threads_started":1}
I20260806 19:46:58.365150 7279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:58.364080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:45843)
0806 19:46:58.364754 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:58.364958 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":334,"threads_started":1}
I20260806 19:46:58.861806 7280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:58.860881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:44541)
0806 19:46:58.861458 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:58.861564 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":297,"threads_started":1}
I20260806 19:46:58.865989 7281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:58.865130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:37261)
0806 19:46:58.865695 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:58.865795 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":243,"threads_started":1}
I20260806 19:46:59.367236 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:59.366345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:54781)
0806 19:46:59.366916 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:59.367038 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":311,"threads_started":1}
I20260806 19:46:59.864483 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:59.863293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:34801)
0806 19:46:59.864151 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:59.864303 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":555,"threads_started":1}
I20260806 19:46:59.868235 7284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:46:59.867155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:58261)
0806 19:46:59.867891 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0806 19:46:59.868009 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":399,"threads_started":1}
I20260806 19:47:00.369269 7285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:00.368235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:35991)
0806 19:47:00.368914 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:00.369057 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":340,"threads_started":1}
I20260806 19:47:00.866628 7286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:00.865665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:50759)
0806 19:47:00.866255 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:00.866371 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":266,"threads_started":1}
I20260806 19:47:00.870175 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:00.869347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:44259)
0806 19:47:00.869913 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:00.870003 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":318,"threads_started":1}
I20260806 19:47:01.372025 7288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:01.370541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:49509)
0806 19:47:01.371510 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:01.371656 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":440,"threads_started":1}
I20260806 19:47:01.868302 7289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:01.867594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:41951)
0806 19:47:01.868034 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:01.868121 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":232,"threads_started":1}
I20260806 19:47:01.872016 7290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:01.871162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:34823)
0806 19:47:01.871725 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:01.871803 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":293,"threads_started":1}
I20260806 19:47:02.377246 7291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:02.376267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:35833)
0806 19:47:02.376977 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:02.377073 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":262,"threads_started":1}
I20260806 19:47:02.870297 7292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:02.869494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:53695)
0806 19:47:02.870020 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:02.870112 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":252,"threads_started":1}
I20260806 19:47:02.873934 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:02.873055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:43547)
0806 19:47:02.873688 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:02.873789 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":243,"threads_started":1}
I20260806 19:47:03.380142 7294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:03.378817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:34265)
0806 19:47:03.379710 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:03.379859 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":393,"threads_started":1}
I20260806 19:47:03.872571 7295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:03.871671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:55415)
0806 19:47:03.872311 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:03.872401 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":365,"threads_started":1}
I20260806 19:47:03.875803 7296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:03.874866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:49289)
0806 19:47:03.875501 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:03.875602 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":324,"threads_started":1}
I20260806 19:47:04.382282 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:04.381463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:53093)
0806 19:47:04.382014 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:04.382111 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":307,"threads_started":1}
I20260806 19:47:04.874667 7298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:04.873648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:55387)
0806 19:47:04.874209 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:04.874337 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":306,"threads_started":1}
I20260806 19:47:04.878203 7299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:04.876899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:41723)
0806 19:47:04.877847 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:04.877982 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":525,"threads_started":1}
I20260806 19:47:05.384157 7300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:05.383372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:35675)
0806 19:47:05.383850 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:05.383947 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":230,"threads_started":1}
I20260806 19:47:05.876359 7301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:05.875630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:42983)
0806 19:47:05.876126 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:05.876209 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":285,"threads_started":1}
I20260806 19:47:05.880249 7302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:05.879359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:49101)
0806 19:47:05.879927 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:05.880056 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":294,"threads_started":1}
I20260806 19:47:06.386111 7303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:06.385187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:38669)
0806 19:47:06.385726 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:06.385885 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":679,"threads_started":1}
I20260806 19:47:06.878317 7304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:06.877429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:38833)
0806 19:47:06.877995 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:06.878126 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":337,"threads_started":1}
I20260806 19:47:06.881814 7305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:06.881155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:49647)
0806 19:47:06.881584 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:06.881661 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":204,"threads_started":1}
I20260806 19:47:07.388574 7306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:07.387540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:39147)
0806 19:47:07.388230 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:07.388357 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":286,"threads_started":1}
I20260806 19:47:07.880518 7307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:07.879669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:56829)
0806 19:47:07.880256 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:07.880339 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":344,"threads_started":1}
I20260806 19:47:07.883457 7308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:07.882650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:58753)
0806 19:47:07.883180 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:07.883278 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":290,"threads_started":1}
I20260806 19:47:08.390425 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:08.389642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:54279)
0806 19:47:08.390123 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:08.390214 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":239,"threads_started":1}
I20260806 19:47:08.882649 7310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:08.881776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:43455)
0806 19:47:08.882350 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:08.882455 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":287,"threads_started":1}
I20260806 19:47:08.885193 7311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:08.884355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:50699)
0806 19:47:08.884933 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:08.885017 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":255,"threads_started":1}
I20260806 19:47:09.392580 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:09.391768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.66:49697)
0806 19:47:09.392267 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:09.392389 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":273,"threads_started":1}
I20260806 19:47:09.865327 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 6834
I20260806 19:47:09.885071 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:09.883958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.65:54157)
0806 19:47:09.884593 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:09.884748 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":339,"threads_started":1}
I20260806 19:47:09.887878 7313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:09.866892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.21.202.65:41599)
0806 19:47:09.867556 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:09.867572 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:09.867642 (+ 70us) server_negotiation.cc:407] Connection header received
0806 19:47:09.869738 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:09.869749 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:09.870209 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:09.870540 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:09.872230 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:09.873770 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:09.875838 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:09.876591 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:09.876753 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:09.880621 (+ 3868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:09.880630 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:09.880648 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:09.880690 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:09.884174 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:09.885040 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:09.885047 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:09.885054 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:09.885267 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:09.885669 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:09.885675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:09.885678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:09.886279 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:09.886638 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:09.887110 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 19:47:09.887596 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":312,"threads_started":1}
I20260806 19:47:09.887900 7315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:09.886875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46849 (local address 127.21.202.67:38363)
0806 19:47:09.887596 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:09.887717 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.126:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":338,"threads_started":1}
I20260806 19:47:09.889456 6998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:09.981825 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 6834
I20260806 19:47:10.004695 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 7006
I20260806 19:47:10.026880 7317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:10.005892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32924 (local address 127.21.202.66:33417)
0806 19:47:10.006513 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:10.006529 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:10.006576 (+ 47us) server_negotiation.cc:407] Connection header received
0806 19:47:10.008379 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:10.008391 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:10.009037 (+ 646us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:10.009622 (+ 585us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:10.011722 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:10.013282 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:10.014880 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:10.015710 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:10.015944 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:10.019296 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:10.019305 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:10.019324 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:10.019370 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:10.022845 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:10.023723 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:10.023731 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:10.023739 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:10.023948 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:10.024410 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:10.024423 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:10.024426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:10.025135 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:10.025503 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:10.026028 (+ 525us) server_negotiation.cc:299] Negotiation successful
0806 19:47:10.026561 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":280,"threads_started":1}
I20260806 19:47:10.028602 7129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:10.121054 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 7006
I20260806 19:47:10.141589 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 7137
I20260806 19:47:10.162885 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:10.142726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.21.202.67:39295)
0806 19:47:10.143401 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:10.143417 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:10.143460 (+ 43us) server_negotiation.cc:407] Connection header received
0806 19:47:10.145442 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:10.145455 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:10.145922 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:10.146318 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:10.148066 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:10.149531 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:10.150851 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:10.151553 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:10.151768 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:10.154786 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:10.154795 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:10.154813 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:10.154865 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:10.158686 (+ 3821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:10.159654 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:10.159662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:10.159671 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:10.159875 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:10.160331 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:10.160339 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:10.160347 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:10.161105 (+ 758us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:10.161447 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:10.162010 (+ 563us) server_negotiation.cc:299] Negotiation successful
0806 19:47:10.162519 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":331,"threads_started":1}
I20260806 19:47:10.164510 7260 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:10.252265 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 7137
2026-08-06T19:47:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (66331 ms)
[----------] 30 tests from SecurityITest (319873 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:10 dist-test-slave-77v9 krb5kdc[7323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:10 dist-test-slave-77v9 krb5kdc[7323](info): set up 2 sockets
Aug 06 19:47:10 dist-test-slave-77v9 krb5kdc[7323](info): commencing operation
krb5kdc: starting...
W20260806 19:47:11.442004 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:11 dist-test-slave-77v9 krb5kdc[7323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:47:11.781760 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46709
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:46709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:12.149951 7336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:12.150293 7336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:12.150393 7336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:12.158288 7336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:12.158439 7336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:12.158491 7336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:12.158543 7336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:12.158592 7336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:12.169337 7336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:46709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:12.170745 7336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:12.172031 7336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:12.185398 7342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:12.185364 7344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:12.185571 7341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:12.186148 7336 server_base.cc:1034] running on GCE node
I20260806 19:47:12.187047 7336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:12.188567 7336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:12.189821 7336 hybrid_clock.cc:648] HybridClock initialized: now 1786045632189774 us; error 56 us; skew 500 ppm
I20260806 19:47:12.190244 7336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:12 dist-test-slave-77v9 krb5kdc[7323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:12.193713 7336 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:47:12.195389 7336 webserver.cc:460] Webserver started at http://127.21.202.126:35971/ using document root <none> and password file <none>
I20260806 19:47:12.196106 7336 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:12.196241 7336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:12.196575 7336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:12.199767 7336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "3a1fb66181934ef6bdacbc6226f5439a"
format_stamp: "Formatted at 2026-08-06 19:47:12 on dist-test-slave-77v9"
server_key: "3f4c769342e8161c7f5519bed2b98873"
server_key_iv: "cb1741983b978e9abbde3e323399bd6b"
server_key_version: "encryptionkey@0"
I20260806 19:47:12.200553 7336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "3a1fb66181934ef6bdacbc6226f5439a"
format_stamp: "Formatted at 2026-08-06 19:47:12 on dist-test-slave-77v9"
server_key: "3f4c769342e8161c7f5519bed2b98873"
server_key_iv: "cb1741983b978e9abbde3e323399bd6b"
server_key_version: "encryptionkey@0"
I20260806 19:47:12.206313 7336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 19:47:12.210166 7351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:12.211556 7336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:47:12.211776 7336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "3a1fb66181934ef6bdacbc6226f5439a"
format_stamp: "Formatted at 2026-08-06 19:47:12 on dist-test-slave-77v9"
server_key: "3f4c769342e8161c7f5519bed2b98873"
server_key_iv: "cb1741983b978e9abbde3e323399bd6b"
server_key_version: "encryptionkey@0"
I20260806 19:47:12.212028 7336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:12.233501 7336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:12.238281 7336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:12.238627 7336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:12.260589 7336 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:46709
I20260806 19:47:12.260612 7403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:46709 every 8 connection(s)
I20260806 19:47:12.262183 7336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:12.265398 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 7336
I20260806 19:47:12.265641 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:12.266541 22313 external_mini_cluster.cc:1428] Setting key 15665cb968c23c36557f3394f893a259
I20260806 19:47:12.267230 7404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:12.283516 7404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a: Bootstrap starting.
Aug 06 19:47:12 dist-test-slave-77v9 krb5kdc[7323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:47:12.288127 7404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:12.289839 7404 log.cc:824] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:12.294032 7404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a: No bootstrap required, opened a new log
I20260806 19:47:12.295414 7407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:12.268744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.21.202.126:46709)
0806 19:47:12.269616 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:12.269628 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:12.269668 (+ 40us) server_negotiation.cc:407] Connection header received
0806 19:47:12.271959 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:12.271984 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:12.272392 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:12.272733 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:12.274862 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:12.276228 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:12.278439 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:12.279262 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:12.279444 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:12.285045 (+ 5601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:12.285060 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:12.285080 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:12.285137 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:12.290048 (+ 4911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:12.291086 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:12.291094 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:12.291100 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:12.291298 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:12.291762 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:12.291767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:12.291773 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:12.292534 (+ 761us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:12.292930 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:12.293381 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 19:47:12.293955 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":378,"threads_started":1}
I20260806 19:47:12.302796 7404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1fb66181934ef6bdacbc6226f5439a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46709 } }
I20260806 19:47:12.303236 7404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:12.303347 7404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1fb66181934ef6bdacbc6226f5439a, State: Initialized, Role: FOLLOWER
I20260806 19:47:12.303996 7404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1fb66181934ef6bdacbc6226f5439a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46709 } }
I20260806 19:47:12.304247 7404 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:12.304396 7404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:12.304575 7404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:12.307143 7404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1fb66181934ef6bdacbc6226f5439a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46709 } }
I20260806 19:47:12.307834 7404 leader_election.cc:302] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a1fb66181934ef6bdacbc6226f5439a; no voters:
I20260806 19:47:12.308607 7404 leader_election.cc:288] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:12.308718 7409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:12.309808 7409 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [term 1 LEADER]: Becoming Leader. State: Replica: 3a1fb66181934ef6bdacbc6226f5439a, State: Running, Role: LEADER
I20260806 19:47:12.310349 7409 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1fb66181934ef6bdacbc6226f5439a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46709 } }
I20260806 19:47:12.311205 7404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:12.313993 7410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a1fb66181934ef6bdacbc6226f5439a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1fb66181934ef6bdacbc6226f5439a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46709 } } }
I20260806 19:47:12.314433 7410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:12.314927 7411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a1fb66181934ef6bdacbc6226f5439a. Latest consensus state: current_term: 1 leader_uuid: "3a1fb66181934ef6bdacbc6226f5439a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1fb66181934ef6bdacbc6226f5439a" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46709 } } }
I20260806 19:47:12.315312 7411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:12.316092 7417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:12.321966 7417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:12.332367 7417 catalog_manager.cc:1347] Generated new cluster ID: f6354ec29c0c4a26894987818409561f
I20260806 19:47:12.332506 7417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:12.354108 7417 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:12.355517 7417 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:12.366189 7417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a: Generated new TSK 0
I20260806 19:47:12.367174 7417 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:47:12.453191 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46709
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:12.810631 7430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:12.810936 7430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:12.811029 7430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:12.818856 7430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:12.818986 7430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:12.819103 7430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:47:12.829419 7430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:12.830760 7430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:12.832037 7430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:12.846870 7435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:12.846930 7436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:12.846899 7438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:12.847779 7430 server_base.cc:1034] running on GCE node
I20260806 19:47:12.848668 7430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:12.849745 7430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:12.851008 7430 hybrid_clock.cc:648] HybridClock initialized: now 1786045632850955 us; error 40 us; skew 500 ppm
I20260806 19:47:12.851382 7430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:12 dist-test-slave-77v9 krb5kdc[7323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:12.855537 7430 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:47:12.857888 7430 webserver.cc:460] Webserver started at http://127.21.202.65:37927/ using document root <none> and password file <none>
I20260806 19:47:12.858850 7430 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:12.859025 7430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:12.859422 7430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:12.862995 7430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "d8b131b7d268460e8b5529cc4ef8edd7"
format_stamp: "Formatted at 2026-08-06 19:47:12 on dist-test-slave-77v9"
server_key: "62c24be35e8bef0d416ae1546638478c"
server_key_iv: "633511d9d55ee4766150e53a1eb775f1"
server_key_version: "encryptionkey@0"
I20260806 19:47:12.863940 7430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "d8b131b7d268460e8b5529cc4ef8edd7"
format_stamp: "Formatted at 2026-08-06 19:47:12 on dist-test-slave-77v9"
server_key: "62c24be35e8bef0d416ae1546638478c"
server_key_iv: "633511d9d55ee4766150e53a1eb775f1"
server_key_version: "encryptionkey@0"
I20260806 19:47:12.870435 7430 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260806 19:47:12.874255 7445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:12.876000 7430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:47:12.876212 7430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "d8b131b7d268460e8b5529cc4ef8edd7"
format_stamp: "Formatted at 2026-08-06 19:47:12 on dist-test-slave-77v9"
server_key: "62c24be35e8bef0d416ae1546638478c"
server_key_iv: "633511d9d55ee4766150e53a1eb775f1"
server_key_version: "encryptionkey@0"
I20260806 19:47:12.876438 7430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:12.901443 7430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:12.906328 7430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:12.906672 7430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:12.907747 7430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:12.909225 7430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:12.909308 7430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:12.909441 7430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:12.909508 7430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:12.945041 7430 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:38127
I20260806 19:47:12.945160 7557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:38127 every 8 connection(s)
I20260806 19:47:12.946473 7430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:47:12.946941 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 7430
I20260806 19:47:12.947234 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:47:12.948135 22313 external_mini_cluster.cc:1428] Setting key 48e861c974a1c5276b40cb7e4c126da6
Aug 06 19:47:12 dist-test-slave-77v9 krb5kdc[7323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:47:12.969460 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:12.949550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:57747 (local address 127.21.202.126:46709)
0806 19:47:12.952982 (+ 3432us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:12.952991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:12.953017 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:47:12.953245 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:12.953253 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:12.953345 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:12.953466 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:12.955033 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:12.956204 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:12.957909 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:12.958603 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:12.958767 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:12.962571 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:12.962576 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:12.962580 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:12.962618 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:12.965414 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:12.966478 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:12.966482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:12.966484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:12.966620 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:12.967189 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:12.967192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:12.967194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:12.967507 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:12.967626 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:12.968992 (+ 1366us) server_negotiation.cc:299] Negotiation successful
0806 19:47:12.969185 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":424,"threads_started":1}
I20260806 19:47:12.971042 7561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:12.950277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46709 (local address 127.21.202.65:57747)
0806 19:47:12.951135 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:12.951182 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:12.953003 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:12.953662 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:12.953669 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:12.954275 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:12.954857 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:12.954873 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:12.956380 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:12.956385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:12.957746 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:12.957759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:12.958826 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:12.958831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:12.959028 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:12.959740 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:47:12.959762 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:12.962300 (+ 2538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:12.965582 (+ 3282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:12.965595 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:12.965616 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:12.966309 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:12.966762 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:12.966769 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:12.966776 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:12.967049 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:12.967643 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:12.967654 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:12.968150 (+ 496us) client_negotiation.cc:769] Sending connection context
0806 19:47:12.968768 (+ 618us) client_negotiation.cc:240] Negotiation successful
0806 19:47:12.969441 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":360,"threads_started":1}
I20260806 19:47:12.972916 7558 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46709
I20260806 19:47:12.973281 7558 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:12.974063 7558 heartbeater.cc:511] Master 127.21.202.126:46709 requested a full tablet report, sending...
I20260806 19:47:12.976404 7369 ts_manager.cc:194] Registered new tserver with Master: d8b131b7d268460e8b5529cc4ef8edd7 (127.21.202.65:38127)
I20260806 19:47:12.978053 7369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:57747
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:47:13.020864 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46709
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:13.383365 7564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:13.383666 7564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:13.383834 7564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:13.392655 7564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:13.392959 7564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:13.393169 7564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:47:13.404542 7564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:13.406013 7564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:13.407397 7564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:13.421427 7569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:13.421505 7570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:13.421481 7572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:13.422959 7564 server_base.cc:1034] running on GCE node
I20260806 19:47:13.423621 7564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:13.424721 7564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:13.426101 7564 hybrid_clock.cc:648] HybridClock initialized: now 1786045633426052 us; error 36 us; skew 500 ppm
I20260806 19:47:13.426499 7564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:13 dist-test-slave-77v9 krb5kdc[7323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045633, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:13.439039 7564 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:47:13.440861 7564 webserver.cc:460] Webserver started at http://127.21.202.66:38559/ using document root <none> and password file <none>
I20260806 19:47:13.441666 7564 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:13.441782 7564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:13.442116 7564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:13.445514 7564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "5c7df15117d242788a22afdcf540b59b"
format_stamp: "Formatted at 2026-08-06 19:47:13 on dist-test-slave-77v9"
server_key: "30f82c2c3da1bc26988037b0a7caf6da"
server_key_iv: "760a83e86170dfcee7758a23526aa169"
server_key_version: "encryptionkey@0"
I20260806 19:47:13.446375 7564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "5c7df15117d242788a22afdcf540b59b"
format_stamp: "Formatted at 2026-08-06 19:47:13 on dist-test-slave-77v9"
server_key: "30f82c2c3da1bc26988037b0a7caf6da"
server_key_iv: "760a83e86170dfcee7758a23526aa169"
server_key_version: "encryptionkey@0"
I20260806 19:47:13.452036 7564 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 19:47:13.455866 7579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:13.457523 7564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:47:13.457723 7564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "5c7df15117d242788a22afdcf540b59b"
format_stamp: "Formatted at 2026-08-06 19:47:13 on dist-test-slave-77v9"
server_key: "30f82c2c3da1bc26988037b0a7caf6da"
server_key_iv: "760a83e86170dfcee7758a23526aa169"
server_key_version: "encryptionkey@0"
I20260806 19:47:13.457957 7564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:13.474784 7564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:13.479645 7564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:13.479996 7564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:13.481254 7564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:13.482681 7564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:13.482795 7564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:13.482971 7564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:13.483047 7564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:13.521535 7564 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:33245
I20260806 19:47:13.521576 7692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:33245 every 8 connection(s)
I20260806 19:47:13.522892 7564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:47:13.524923 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 7564
I20260806 19:47:13.525125 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:47:13.525810 22313 external_mini_cluster.cc:1428] Setting key 1ad20606178b960cb2aa1d9a8de0dcf0
Aug 06 19:47:13 dist-test-slave-77v9 krb5kdc[7323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045633, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:47:13.547839 7695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:13.525938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:51843 (local address 127.21.202.126:46709)
0806 19:47:13.526489 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:13.526496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:13.527763 (+ 1267us) server_negotiation.cc:407] Connection header received
0806 19:47:13.529962 (+ 2199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:13.529972 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:13.530072 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:13.530206 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:13.532071 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:13.533235 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:13.535708 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:13.536489 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:13.536648 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:13.541432 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:13.541437 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:13.541440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:13.541474 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:13.544219 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:13.545123 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:13.545129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:13.545134 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:13.545278 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:13.545907 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:13.545915 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:13.545924 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:13.546215 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:13.546381 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:13.547178 (+ 797us) server_negotiation.cc:299] Negotiation successful
0806 19:47:13.547448 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":259,"threads_started":1}
I20260806 19:47:13.548929 7696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:13.526586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46709 (local address 127.21.202.66:51843)
0806 19:47:13.527564 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:13.527620 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:13.529635 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:13.530497 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:13.530506 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:13.531070 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:13.531865 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:13.531889 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:13.533390 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:13.533398 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:13.535524 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:13.535542 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:13.536729 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:13.536737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:13.537083 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:13.537945 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:47:13.537978 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:13.541124 (+ 3146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:13.544403 (+ 3279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:13.544413 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:13.544432 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:13.544995 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:13.545537 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:13.545544 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:13.545548 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:13.545772 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:13.546355 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:13.546364 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:13.546763 (+ 399us) client_negotiation.cc:769] Sending connection context
0806 19:47:13.547065 (+ 302us) client_negotiation.cc:240] Negotiation successful
0806 19:47:13.547649 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":401,"threads_started":1}
I20260806 19:47:13.550604 7693 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46709
I20260806 19:47:13.550990 7693 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:13.551759 7693 heartbeater.cc:511] Master 127.21.202.126:46709 requested a full tablet report, sending...
I20260806 19:47:13.553393 7369 ts_manager.cc:194] Registered new tserver with Master: 5c7df15117d242788a22afdcf540b59b (127.21.202.66:33245)
I20260806 19:47:13.554335 7369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:51843
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:47:13.601528 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46709
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:13.966351 7700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:13.966732 7700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:13.966918 7700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:13.974524 7700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:13.974701 7700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:13.974897 7700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:47:13.981690 7558 heartbeater.cc:503] Master 127.21.202.126:46709 was elected leader, sending a full tablet report...
I20260806 19:47:13.986624 7700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:13.987982 7700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:13.989413 7700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:14.003336 7706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:14.003432 7708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:14.003405 7705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:14.004456 7700 server_base.cc:1034] running on GCE node
I20260806 19:47:14.005192 7700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:14.006278 7700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:14.007561 7700 hybrid_clock.cc:648] HybridClock initialized: now 1786045634007511 us; error 47 us; skew 500 ppm
I20260806 19:47:14.007958 7700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:14 dist-test-slave-77v9 krb5kdc[7323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045634, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:14.012475 7700 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:47:14.014721 7700 webserver.cc:460] Webserver started at http://127.21.202.67:43785/ using document root <none> and password file <none>
I20260806 19:47:14.015597 7700 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:14.015756 7700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:14.016170 7700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:14.019666 7700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "53f32038e44a4d499c207eb1c6681130"
format_stamp: "Formatted at 2026-08-06 19:47:14 on dist-test-slave-77v9"
server_key: "c98859154d42a52a88f94a34b47244fe"
server_key_iv: "f70b8ce7fb566f957733839507c430df"
server_key_version: "encryptionkey@0"
I20260806 19:47:14.020542 7700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "53f32038e44a4d499c207eb1c6681130"
format_stamp: "Formatted at 2026-08-06 19:47:14 on dist-test-slave-77v9"
server_key: "c98859154d42a52a88f94a34b47244fe"
server_key_iv: "f70b8ce7fb566f957733839507c430df"
server_key_version: "encryptionkey@0"
I20260806 19:47:14.027045 7700 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260806 19:47:14.031144 7715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:14.032796 7700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:47:14.033090 7700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "53f32038e44a4d499c207eb1c6681130"
format_stamp: "Formatted at 2026-08-06 19:47:14 on dist-test-slave-77v9"
server_key: "c98859154d42a52a88f94a34b47244fe"
server_key_iv: "f70b8ce7fb566f957733839507c430df"
server_key_version: "encryptionkey@0"
I20260806 19:47:14.033366 7700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:14.049850 7700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:14.055502 7700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:14.055859 7700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:14.057387 7700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:14.059052 7700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:14.059139 7700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:14.059297 7700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:14.059384 7700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:14.099407 7700 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:40283
I20260806 19:47:14.099459 7828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:40283 every 8 connection(s)
I20260806 19:47:14.100775 7700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:47:14.105265 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 7700
I20260806 19:47:14.105481 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:47:14.106079 22313 external_mini_cluster.cc:1428] Setting key e3a2733f67688f00a2d3601e9e586ed4
Aug 06 19:47:14 dist-test-slave-77v9 krb5kdc[7323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045634, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:47:14.124081 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.103528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:38283 (local address 127.21.202.126:46709)
0806 19:47:14.104073 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:14.104078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:14.105228 (+ 1150us) server_negotiation.cc:407] Connection header received
0806 19:47:14.107881 (+ 2653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:14.107885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:14.107960 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:14.108069 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:14.110096 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.110901 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.112606 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.113139 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.113243 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.117110 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:14.117116 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:14.117120 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:14.117204 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:14.119987 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:14.121306 (+ 1319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:14.121319 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:14.121322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:14.121479 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:14.122103 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:14.122111 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:14.122119 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:14.122444 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:14.122618 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:14.123411 (+ 793us) server_negotiation.cc:299] Negotiation successful
0806 19:47:14.123666 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":287,"threads_started":1}
I20260806 19:47:14.125633 7832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.104093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46709 (local address 127.21.202.67:38283)
0806 19:47:14.105018 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:14.105084 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:14.107498 (+ 2414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:14.108382 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:14.108397 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:14.109132 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:14.109928 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:14.109951 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.111054 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.111057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:14.112456 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:14.112469 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.113362 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.113367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:14.113578 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.114310 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:47:14.114334 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:14.116802 (+ 2468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:14.120210 (+ 3408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:14.120219 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:14.120260 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:14.121088 (+ 828us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:14.121622 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:14.121627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:14.121630 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:14.121964 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:14.122587 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:14.122596 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:14.122989 (+ 393us) client_negotiation.cc:769] Sending connection context
0806 19:47:14.123277 (+ 288us) client_negotiation.cc:240] Negotiation successful
0806 19:47:14.123872 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":318,"threads_started":1}
I20260806 19:47:14.127339 7829 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46709
I20260806 19:47:14.127712 7829 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:14.128533 7829 heartbeater.cc:511] Master 127.21.202.126:46709 requested a full tablet report, sending...
I20260806 19:47:14.130290 7369 ts_manager.cc:194] Registered new tserver with Master: 53f32038e44a4d499c207eb1c6681130 (127.21.202.67:40283)
I20260806 19:47:14.131112 7369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:38283
I20260806 19:47:14.135159 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:47:14.154426 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.140126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35656 (local address 127.21.202.126:46709)
0806 19:47:14.140323 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:14.140336 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:14.140789 (+ 453us) server_negotiation.cc:407] Connection header received
0806 19:47:14.141017 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:14.141021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:14.141091 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:14.141182 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:14.143019 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.143796 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.145403 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.145804 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.145942 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.148504 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:14.148510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:14.148514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:14.148568 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:14.151474 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:14.152250 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:14.152254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:14.152257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:14.152394 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:14.152908 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:14.152912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:14.152914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:14.153286 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:14.153498 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:14.153977 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 19:47:14.154178 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 19:47:14.159565 7369 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35656:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:47:14.167659 7369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:47:14.197479 7843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.182306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:40283 (local address 127.0.0.1:37432)
0806 19:47:14.183891 (+ 1585us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:14.183941 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:14.184166 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:14.186192 (+ 2026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:14.186203 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:14.186243 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:14.186860 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:14.186880 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.189841 (+ 2961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.189850 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:14.193573 (+ 3723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:14.193605 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.196159 (+ 2554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.196165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:14.196398 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.196465 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 19:47:14.196640 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 19:47:14.196937 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1245,"mutex_wait_us":94,"thread_start_us":1057,"threads_started":1}
I20260806 19:47:14.197486 7845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.182703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37432 (local address 127.21.202.67:40283)
0806 19:47:14.185419 (+ 2716us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:14.185430 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:14.185465 (+ 35us) server_negotiation.cc:407] Connection header received
0806 19:47:14.185563 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:14.185570 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:14.185896 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:14.187117 (+ 1221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:14.187457 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.189608 (+ 2151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.193817 (+ 4209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.195602 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.195746 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.196004 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:14.196710 (+ 706us) server_negotiation.cc:299] Negotiation successful
0806 19:47:14.197062 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2539,"thread_start_us":2380,"threads_started":1}
I20260806 19:47:14.199291 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.182275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42722 (local address 127.21.202.66:33245)
0806 19:47:14.183167 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:14.183180 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:14.185539 (+ 2359us) server_negotiation.cc:407] Connection header received
0806 19:47:14.185884 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:14.185892 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:14.186106 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:14.186274 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:14.187427 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.189602 (+ 2175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.195483 (+ 5881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.198164 (+ 2681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.198309 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.198493 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:14.198805 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 19:47:14.199004 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":574,"threads_started":1}
I20260806 19:47:14.199958 7844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.182352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:33245 (local address 127.0.0.1:42722)
0806 19:47:14.185400 (+ 3048us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:14.185415 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:14.185656 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:14.186625 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:14.186629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:14.186663 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:14.187184 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:14.187200 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.189849 (+ 2649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.189858 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:14.195249 (+ 5391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:14.195268 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.198373 (+ 3105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.198379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:14.198567 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.198610 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 19:47:14.199539 (+ 929us) client_negotiation.cc:240] Negotiation successful
0806 19:47:14.199651 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2924,"mutex_wait_us":2721,"thread_start_us":2773,"threads_started":1}
I20260806 19:47:14.201669 7763 tablet_service.cc:1467] Processing CreateTablet for tablet 235bc407e5e94d3ea0197b193e5df438 (DEFAULT_TABLE table=test-table [id=d28a2417e6924e1a8cdbf894ea092083]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:14.203413 7763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 235bc407e5e94d3ea0197b193e5df438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:14.203404 7627 tablet_service.cc:1467] Processing CreateTablet for tablet 235bc407e5e94d3ea0197b193e5df438 (DEFAULT_TABLE table=test-table [id=d28a2417e6924e1a8cdbf894ea092083]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:14.204530 7627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 235bc407e5e94d3ea0197b193e5df438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:14.208570 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.185548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:38127 (local address 127.0.0.1:55286)
0806 19:47:14.197050 (+ 11502us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:14.197079 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:14.197529 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:14.198726 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:14.198732 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:14.198762 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:14.201586 (+ 2824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:14.201599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.203627 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.203632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:14.205720 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:14.205737 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.207706 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.207714 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:14.207893 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.207938 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 19:47:14.208037 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 19:47:14.208189 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11247,"thread_start_us":11718,"threads_started":1}
I20260806 19:47:14.208889 7847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.185982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55286 (local address 127.21.202.65:38127)
0806 19:47:14.193446 (+ 7464us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:14.193458 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:14.197207 (+ 3749us) server_negotiation.cc:407] Connection header received
0806 19:47:14.198200 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:14.198209 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:14.198501 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:14.198755 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:14.201790 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.203410 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.205913 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.207472 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.207622 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.207844 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:14.208138 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 19:47:14.208331 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7275,"thread_start_us":7114,"threads_started":1}
I20260806 19:47:14.213047 7493 tablet_service.cc:1467] Processing CreateTablet for tablet 235bc407e5e94d3ea0197b193e5df438 (DEFAULT_TABLE table=test-table [id=d28a2417e6924e1a8cdbf894ea092083]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:14.216403 7493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 235bc407e5e94d3ea0197b193e5df438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:14.216894 7849 tablet_bootstrap.cc:492] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b: Bootstrap starting.
I20260806 19:47:14.218545 7850 tablet_bootstrap.cc:492] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130: Bootstrap starting.
I20260806 19:47:14.221211 7849 tablet_bootstrap.cc:654] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:14.222138 7850 tablet_bootstrap.cc:654] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:14.222563 7849 log.cc:824] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:14.223483 7850 log.cc:824] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:14.226511 7849 tablet_bootstrap.cc:492] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b: No bootstrap required, opened a new log
I20260806 19:47:14.226830 7849 ts_tablet_manager.cc:1397] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260806 19:47:14.226977 7850 tablet_bootstrap.cc:492] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130: No bootstrap required, opened a new log
I20260806 19:47:14.227244 7850 ts_tablet_manager.cc:1397] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 19:47:14.231698 7853 tablet_bootstrap.cc:492] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7: Bootstrap starting.
I20260806 19:47:14.236129 7853 tablet_bootstrap.cc:654] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:14.235996 7849 raft_consensus.cc:348] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7df15117d242788a22afdcf540b59b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33245 } } peers { permanent_uuid: "53f32038e44a4d499c207eb1c6681130" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40283 } } peers { permanent_uuid: "d8b131b7d268460e8b5529cc4ef8edd7" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38127 } }
I20260806 19:47:14.236514 7849 raft_consensus.cc:374] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:14.236637 7849 raft_consensus.cc:729] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7df15117d242788a22afdcf540b59b, State: Initialized, Role: FOLLOWER
I20260806 19:47:14.237399 7849 consensus_queue.cc:260] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7df15117d242788a22afdcf540b59b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33245 } } peers { permanent_uuid: "53f32038e44a4d499c207eb1c6681130" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40283 } } peers { permanent_uuid: "d8b131b7d268460e8b5529cc4ef8edd7" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38127 } }
I20260806 19:47:14.237609 7853 log.cc:824] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:14.238678 7693 heartbeater.cc:503] Master 127.21.202.126:46709 was elected leader, sending a full tablet report...
I20260806 19:47:14.238845 7850 raft_consensus.cc:348] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7df15117d242788a22afdcf540b59b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33245 } } peers { permanent_uuid: "53f32038e44a4d499c207eb1c6681130" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40283 } } peers { permanent_uuid: "d8b131b7d268460e8b5529cc4ef8edd7" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38127 } }
I20260806 19:47:14.239303 7850 raft_consensus.cc:374] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:14.239395 7850 raft_consensus.cc:729] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f32038e44a4d499c207eb1c6681130, State: Initialized, Role: FOLLOWER
I20260806 19:47:14.239692 7849 ts_tablet_manager.cc:1428] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260806 19:47:14.240015 7850 consensus_queue.cc:260] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7df15117d242788a22afdcf540b59b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33245 } } peers { permanent_uuid: "53f32038e44a4d499c207eb1c6681130" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40283 } } peers { permanent_uuid: "d8b131b7d268460e8b5529cc4ef8edd7" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38127 } }
I20260806 19:47:14.241643 7829 heartbeater.cc:503] Master 127.21.202.126:46709 was elected leader, sending a full tablet report...
I20260806 19:47:14.242210 7850 ts_tablet_manager.cc:1428] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130: Time spent starting tablet: real 0.015s user 0.013s sys 0.003s
I20260806 19:47:14.245977 7856 raft_consensus.cc:482] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:47:14.246292 7856 raft_consensus.cc:504] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7df15117d242788a22afdcf540b59b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33245 } } peers { permanent_uuid: "53f32038e44a4d499c207eb1c6681130" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40283 } } peers { permanent_uuid: "d8b131b7d268460e8b5529cc4ef8edd7" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38127 } }
I20260806 19:47:14.247829 7853 tablet_bootstrap.cc:492] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7: No bootstrap required, opened a new log
I20260806 19:47:14.248175 7853 ts_tablet_manager.cc:1397] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.000s
I20260806 19:47:14.248412 7856 leader_election.cc:288] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7df15117d242788a22afdcf540b59b (127.21.202.66:33245), d8b131b7d268460e8b5529cc4ef8edd7 (127.21.202.65:38127)
I20260806 19:47:14.255879 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.248924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:50383 (local address 127.21.202.66:33245)
0806 19:47:14.249119 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:14.249125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:14.249151 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:47:14.249309 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:14.249314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:14.249389 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:14.249498 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:14.250207 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.251254 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.253790 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.254950 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.255112 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.255188 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:14.255479 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 19:47:14.255609 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 19:47:14.255872 7832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.248778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:33245 (local address 127.21.202.67:50383)
0806 19:47:14.249012 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:14.249034 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:14.249162 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:14.249888 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:14.249892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:14.249915 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:14.250065 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:14.250072 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.251383 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.251390 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:14.253599 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:14.253615 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.255113 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.255119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:14.255314 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.255340 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 19:47:14.255443 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 19:47:14.255582 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 19:47:14.257236 7847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.249562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:41923 (local address 127.21.202.65:38127)
0806 19:47:14.249808 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:14.249814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:14.250704 (+ 890us) server_negotiation.cc:407] Connection header received
0806 19:47:14.250995 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:14.251000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:14.251074 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:14.251175 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:14.251842 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.252603 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.254761 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.256139 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.256324 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.256401 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:14.256844 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 19:47:14.256992 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 19:47:14.257040 7647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "235bc407e5e94d3ea0197b193e5df438" candidate_uuid: "53f32038e44a4d499c207eb1c6681130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c7df15117d242788a22afdcf540b59b" is_pre_election: true
I20260806 19:47:14.257361 7858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.249173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:38127 (local address 127.21.202.67:41923)
0806 19:47:14.250610 (+ 1437us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:14.250628 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:14.250800 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:14.251212 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:14.251216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:14.251248 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:14.251673 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:14.251681 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.252756 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.252764 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:14.254570 (+ 1806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:14.254586 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.256379 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.256384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:14.256588 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.256627 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 19:47:14.256774 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 19:47:14.256937 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1305,"mutex_wait_us":1,"thread_start_us":1174,"threads_started":1}
I20260806 19:47:14.257637 7647 raft_consensus.cc:2454] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53f32038e44a4d499c207eb1c6681130 in term 0.
I20260806 19:47:14.258460 7717 leader_election.cc:302] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53f32038e44a4d499c207eb1c6681130, 5c7df15117d242788a22afdcf540b59b; no voters:
I20260806 19:47:14.258621 7513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "235bc407e5e94d3ea0197b193e5df438" candidate_uuid: "53f32038e44a4d499c207eb1c6681130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b131b7d268460e8b5529cc4ef8edd7" is_pre_election: true
I20260806 19:47:14.259680 7856 raft_consensus.cc:2781] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:47:14.259815 7856 raft_consensus.cc:482] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:47:14.259950 7856 raft_consensus.cc:3037] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:14.259673 7853 raft_consensus.cc:348] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7df15117d242788a22afdcf540b59b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33245 } } peers { permanent_uuid: "53f32038e44a4d499c207eb1c6681130" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40283 } } peers { permanent_uuid: "d8b131b7d268460e8b5529cc4ef8edd7" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38127 } }
I20260806 19:47:14.260103 7853 raft_consensus.cc:374] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260806 19:47:14.260035 7717 leader_election.cc:341] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8b131b7d268460e8b5529cc4ef8edd7 (127.21.202.65:38127): Illegal state: must be running to vote when last-logged opid is not known
I20260806 19:47:14.260263 7853 raft_consensus.cc:729] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b131b7d268460e8b5529cc4ef8edd7, State: Initialized, Role: FOLLOWER
I20260806 19:47:14.260980 7853 consensus_queue.cc:260] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7df15117d242788a22afdcf540b59b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33245 } } peers { permanent_uuid: "53f32038e44a4d499c207eb1c6681130" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40283 } } peers { permanent_uuid: "d8b131b7d268460e8b5529cc4ef8edd7" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38127 } }
I20260806 19:47:14.262849 7853 ts_tablet_manager.cc:1428] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
I20260806 19:47:14.263289 7856 raft_consensus.cc:504] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7df15117d242788a22afdcf540b59b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33245 } } peers { permanent_uuid: "53f32038e44a4d499c207eb1c6681130" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40283 } } peers { permanent_uuid: "d8b131b7d268460e8b5529cc4ef8edd7" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38127 } }
I20260806 19:47:14.265156 7856 leader_election.cc:288] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [CANDIDATE]: Term 1 election: Requested vote from peers 5c7df15117d242788a22afdcf540b59b (127.21.202.66:33245), d8b131b7d268460e8b5529cc4ef8edd7 (127.21.202.65:38127)
I20260806 19:47:14.265683 7513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "235bc407e5e94d3ea0197b193e5df438" candidate_uuid: "53f32038e44a4d499c207eb1c6681130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b131b7d268460e8b5529cc4ef8edd7"
I20260806 19:47:14.265842 7647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "235bc407e5e94d3ea0197b193e5df438" candidate_uuid: "53f32038e44a4d499c207eb1c6681130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c7df15117d242788a22afdcf540b59b"
I20260806 19:47:14.265954 7513 raft_consensus.cc:3037] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:14.266091 7647 raft_consensus.cc:3037] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:14.269280 7513 raft_consensus.cc:2454] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53f32038e44a4d499c207eb1c6681130 in term 1.
I20260806 19:47:14.269778 7717 leader_election.cc:302] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53f32038e44a4d499c207eb1c6681130, d8b131b7d268460e8b5529cc4ef8edd7; no voters:
I20260806 19:47:14.269802 7647 raft_consensus.cc:2454] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53f32038e44a4d499c207eb1c6681130 in term 1.
I20260806 19:47:14.270138 7856 raft_consensus.cc:2781] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:14.270371 7856 raft_consensus.cc:686] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [term 1 LEADER]: Becoming Leader. State: Replica: 53f32038e44a4d499c207eb1c6681130, State: Running, Role: LEADER
I20260806 19:47:14.271070 7856 consensus_queue.cc:237] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7df15117d242788a22afdcf540b59b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33245 } } peers { permanent_uuid: "53f32038e44a4d499c207eb1c6681130" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40283 } } peers { permanent_uuid: "d8b131b7d268460e8b5529cc4ef8edd7" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38127 } }
W20260806 19:47:14.275187 7694 tablet.cc:2369] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:47:14.275961 7369 catalog_manager.cc:5495] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53f32038e44a4d499c207eb1c6681130 (127.21.202.67). New cstate: current_term: 1 leader_uuid: "53f32038e44a4d499c207eb1c6681130" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7df15117d242788a22afdcf540b59b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53f32038e44a4d499c207eb1c6681130" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 40283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8b131b7d268460e8b5529cc4ef8edd7" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38127 } health_report { overall_health: UNKNOWN } } }
I20260806 19:47:14.297361 7845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.291382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37434 (local address 127.21.202.67:40283)
0806 19:47:14.291650 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:14.291655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:14.291674 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:47:14.291929 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:14.291933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:14.292014 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:14.292147 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:47:14.292510 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.293336 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.294842 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.295265 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.295387 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.295983 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:47:14.296528 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:47:14.296648 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:14.296924 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 19:47:14.297007 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260806 19:47:14.307440 7513 raft_consensus.cc:1261] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7 [term 1 FOLLOWER]: Refusing update from remote peer 53f32038e44a4d499c207eb1c6681130: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:47:14.307500 7647 raft_consensus.cc:1261] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b [term 1 FOLLOWER]: Refusing update from remote peer 53f32038e44a4d499c207eb1c6681130: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:47:14.308658 7857 consensus_queue.cc:1035] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b131b7d268460e8b5529cc4ef8edd7" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 38127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:47:14.309244 7856 consensus_queue.cc:1035] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c7df15117d242788a22afdcf540b59b" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 33245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:47:14.339550 7369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:47:14.339877 7369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:47:14.344094 7369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a1fb66181934ef6bdacbc6226f5439a: Sending DeleteTablet for 3 replicas of tablet 235bc407e5e94d3ea0197b193e5df438
I20260806 19:47:14.345765 7763 tablet_service.cc:1514] Processing DeleteTablet for tablet 235bc407e5e94d3ea0197b193e5df438 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:14 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:37432
I20260806 19:47:14.345781 7627 tablet_service.cc:1514] Processing DeleteTablet for tablet 235bc407e5e94d3ea0197b193e5df438 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:14 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:42722
I20260806 19:47:14.346081 7493 tablet_service.cc:1514] Processing DeleteTablet for tablet 235bc407e5e94d3ea0197b193e5df438 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:14 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:55286
I20260806 19:47:14.348677 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 7430
I20260806 19:47:14.356608 7873 tablet_replica.cc:330] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130: stopping tablet replica
I20260806 19:47:14.357163 7873 raft_consensus.cc:2229] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:47:14.357947 7873 raft_consensus.cc:2258] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:14.358760 7874 tablet_replica.cc:330] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7: stopping tablet replica
I20260806 19:47:14.359205 7874 raft_consensus.cc:2229] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:47:14.359874 7874 raft_consensus.cc:2258] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:14.361497 7873 ts_tablet_manager.cc:1905] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:14.362354 7874 ts_tablet_manager.cc:1905] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:14.369453 7872 tablet_replica.cc:330] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b: stopping tablet replica
I20260806 19:47:14.369988 7872 raft_consensus.cc:2229] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:47:14.370576 7872 raft_consensus.cc:2258] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:14.371999 7873 ts_tablet_manager.cc:1918] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:14.372253 7873 log.cc:1196] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/235bc407e5e94d3ea0197b193e5df438
I20260806 19:47:14.372937 7873 ts_tablet_manager.cc:1939] T 235bc407e5e94d3ea0197b193e5df438 P 53f32038e44a4d499c207eb1c6681130: Deleting consensus metadata
I20260806 19:47:14.373057 7872 ts_tablet_manager.cc:1905] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:14.375105 7354 catalog_manager.cc:4841] TS 53f32038e44a4d499c207eb1c6681130 (127.21.202.67:40283): tablet 235bc407e5e94d3ea0197b193e5df438 (table test-table [id=d28a2417e6924e1a8cdbf894ea092083]) successfully deleted
I20260806 19:47:14.377951 7874 ts_tablet_manager.cc:1918] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:14.378155 7874 log.cc:1196] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/235bc407e5e94d3ea0197b193e5df438
I20260806 19:47:14.378774 7874 ts_tablet_manager.cc:1939] T 235bc407e5e94d3ea0197b193e5df438 P d8b131b7d268460e8b5529cc4ef8edd7: Deleting consensus metadata
I20260806 19:47:14.380317 7872 ts_tablet_manager.cc:1918] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:14.380496 7872 log.cc:1196] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/235bc407e5e94d3ea0197b193e5df438
I20260806 19:47:14.381009 7353 catalog_manager.cc:4841] TS d8b131b7d268460e8b5529cc4ef8edd7 (127.21.202.65:38127): tablet 235bc407e5e94d3ea0197b193e5df438 (table test-table [id=d28a2417e6924e1a8cdbf894ea092083]) successfully deleted
I20260806 19:47:14.381373 7872 ts_tablet_manager.cc:1939] T 235bc407e5e94d3ea0197b193e5df438 P 5c7df15117d242788a22afdcf540b59b: Deleting consensus metadata
Aug 06 19:47:14 dist-test-slave-77v9 krb5kdc[7323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:47:14.383183 7353 catalog_manager.cc:4841] TS 5c7df15117d242788a22afdcf540b59b (127.21.202.66:33245): tablet 235bc407e5e94d3ea0197b193e5df438 (table test-table [id=d28a2417e6924e1a8cdbf894ea092083]) successfully deleted
I20260806 19:47:14.389451 7847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.350312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55294 (local address 127.21.202.65:38127)
0806 19:47:14.350556 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:14.350566 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:14.366033 (+ 15467us) server_negotiation.cc:407] Connection header received
0806 19:47:14.369587 (+ 3554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:14.369593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:14.369702 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:14.369783 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:14.371716 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.372961 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.374972 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.375766 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.375853 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.382967 (+ 7114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:14.382984 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:14.383003 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:14.383057 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:14.386315 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:14.387171 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:14.387188 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:14.387190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:14.387349 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:14.387757 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:14.387763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:14.387780 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:14.388149 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:14.388475 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:14.388835 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 19:47:14.389073 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 19:47:14.390198 7553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:14.487241 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 7430
I20260806 19:47:14.507243 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 7564
Aug 06 19:47:14 dist-test-slave-77v9 krb5kdc[7323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:47:14.527109 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.508189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42728 (local address 127.21.202.66:33245)
0806 19:47:14.508424 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:14.508431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:14.508457 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:47:14.508628 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:14.508637 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:14.508704 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:14.508861 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:14.510116 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.510857 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.512287 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.512749 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.512939 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.519141 (+ 6202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:14.519159 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:14.519177 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:14.519226 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:14.523471 (+ 4245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:14.524468 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:14.524477 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:14.524481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:14.524631 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:14.525084 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:14.525089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:14.525093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:14.525615 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:14.525929 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:14.526260 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 19:47:14.526529 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 19:47:14.527844 7688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:14.622026 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 7564
I20260806 19:47:14.641131 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 7700
Aug 06 19:47:14 dist-test-slave-77v9 krb5kdc[7323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:47:14.662307 7845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:14.642073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37450 (local address 127.21.202.67:40283)
0806 19:47:14.642316 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:14.642322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:14.642346 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:47:14.642436 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:14.642441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:14.642531 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:14.642667 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:14.644144 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.645822 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.647322 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:14.648122 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:14.648289 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:14.654294 (+ 6005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:14.654318 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:14.654346 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:14.654423 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:14.658724 (+ 4301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:14.659656 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:14.659667 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:14.659670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:14.659899 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:14.660446 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:14.660454 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:14.660462 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:14.661031 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:14.661383 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:14.661621 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 19:47:14.661878 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 19:47:14.663241 7824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:14.793725 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 7700
I20260806 19:47:14.817325 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 7336
I20260806 19:47:14.818571 7399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:14.934196 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 7336
2026-08-06T19:47:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4685 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T19:47:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:15Z Disabled control of system clock
I20260806 19:47:15.033145 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:44541
--webserver_interface=127.21.202.126
--webserver_port=0
--builtin_ntp_servers=127.21.202.84:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:44541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 19:47:15.403192 7883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:15.403524 7883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:15.403609 7883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:15.411298 7883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:15.411427 7883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:15.411465 7883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:15.411516 7883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:15.411571 7883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:15.421228 7883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:44541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:44541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:15.422590 7883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:15.424475 7883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:15.440677 7889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:15.441195 7891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:15.441231 7888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:15.442045 7883 server_base.cc:1034] running on GCE node
I20260806 19:47:15.442950 7883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:15.444787 7883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:15.446107 7883 hybrid_clock.cc:648] HybridClock initialized: now 1786045635446060 us; error 34 us; skew 500 ppm
I20260806 19:47:15.446537 7883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:47:15.450079 7883 webserver.cc:460] Webserver started at http://127.21.202.126:41425/ using document root <none> and password file <none>
I20260806 19:47:15.450999 7883 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:15.451177 7883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:15.451552 7883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:15.454978 7883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "a1cb32226d3049e0b263bf014ae5f9c1"
format_stamp: "Formatted at 2026-08-06 19:47:15 on dist-test-slave-77v9"
server_key: "b842c4c4ba2c6ecbb6f82084476a84eb"
server_key_iv: "595cbae5bca08975a9a88ae749c26cd2"
server_key_version: "encryptionkey@0"
I20260806 19:47:15.456029 7883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "a1cb32226d3049e0b263bf014ae5f9c1"
format_stamp: "Formatted at 2026-08-06 19:47:15 on dist-test-slave-77v9"
server_key: "b842c4c4ba2c6ecbb6f82084476a84eb"
server_key_iv: "595cbae5bca08975a9a88ae749c26cd2"
server_key_version: "encryptionkey@0"
I20260806 19:47:15.461966 7883 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 19:47:15.466368 7897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:15.468088 7883 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 19:47:15.468330 7883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "a1cb32226d3049e0b263bf014ae5f9c1"
format_stamp: "Formatted at 2026-08-06 19:47:15 on dist-test-slave-77v9"
server_key: "b842c4c4ba2c6ecbb6f82084476a84eb"
server_key_iv: "595cbae5bca08975a9a88ae749c26cd2"
server_key_version: "encryptionkey@0"
I20260806 19:47:15.468595 7883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:15.496784 7883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:15.497884 7883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:15.498260 7883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:15.519730 7883 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:44541
I20260806 19:47:15.519749 7948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:44541 every 8 connection(s)
I20260806 19:47:15.521193 7883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:15.525085 7949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:15.526695 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 7883
I20260806 19:47:15.526919 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:15.527407 22313 external_mini_cluster.cc:1428] Setting key 9268eeee900644e19cd20aae6d40aec1
I20260806 19:47:15.538291 7949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1: Bootstrap starting.
I20260806 19:47:15.542321 7949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:15.543154 7952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:15.529360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55908 (local address 127.21.202.126:44541)
0806 19:47:15.530541 (+ 1181us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:15.530552 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:15.530602 (+ 50us) server_negotiation.cc:407] Connection header received
0806 19:47:15.532767 (+ 2165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:15.532790 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:15.533312 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:15.533626 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:15.534367 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:15.535861 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:15.537959 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:15.538814 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:15.538963 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:15.540424 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:15.540432 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:15.540446 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:15.541164 (+ 718us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:15.541258 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:15.541305 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:15.541404 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:15.541675 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 19:47:15.542023 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":479,"threads_started":1}
I20260806 19:47:15.543573 7949 log.cc:824] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:15.547252 7949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1: No bootstrap required, opened a new log
I20260806 19:47:15.554937 7949 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1cb32226d3049e0b263bf014ae5f9c1" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 44541 } }
I20260806 19:47:15.555315 7949 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:15.555398 7949 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cb32226d3049e0b263bf014ae5f9c1, State: Initialized, Role: FOLLOWER
I20260806 19:47:15.555917 7949 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1cb32226d3049e0b263bf014ae5f9c1" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 44541 } }
I20260806 19:47:15.556124 7949 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:15.556227 7949 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:15.556354 7949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:15.558625 7949 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1cb32226d3049e0b263bf014ae5f9c1" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 44541 } }
I20260806 19:47:15.559231 7949 leader_election.cc:302] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1cb32226d3049e0b263bf014ae5f9c1; no voters:
I20260806 19:47:15.559962 7949 leader_election.cc:288] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:15.560102 7954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:15.560832 7954 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [term 1 LEADER]: Becoming Leader. State: Replica: a1cb32226d3049e0b263bf014ae5f9c1, State: Running, Role: LEADER
I20260806 19:47:15.561456 7954 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1cb32226d3049e0b263bf014ae5f9c1" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 44541 } }
I20260806 19:47:15.562399 7949 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:15.564759 7956 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1cb32226d3049e0b263bf014ae5f9c1. Latest consensus state: current_term: 1 leader_uuid: "a1cb32226d3049e0b263bf014ae5f9c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1cb32226d3049e0b263bf014ae5f9c1" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 44541 } } }
I20260806 19:47:15.564849 7955 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1cb32226d3049e0b263bf014ae5f9c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1cb32226d3049e0b263bf014ae5f9c1" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 44541 } } }
I20260806 19:47:15.565420 7956 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:15.565488 7955 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:15.567535 7959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:15.573791 7959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:15.583467 7959 catalog_manager.cc:1347] Generated new cluster ID: 08a237c98c5d49959bc72a4d684870d5
I20260806 19:47:15.583613 7959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:15.604795 7959 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:15.606791 7959 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:15.618332 7959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1: Generated new TSK 0
I20260806 19:47:15.619560 7959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:15.633006 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:44541
--builtin_ntp_servers=127.21.202.84:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 19:47:16.003768 7973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:16.004136 7973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:16.004266 7973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:16.011600 7973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:16.011725 7973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:16.011835 7973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:47:16.021358 7973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:44541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:16.022670 7973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:16.024544 7973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:16.038816 7979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:16.038988 7981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:16.039016 7978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:16.039673 7973 server_base.cc:1034] running on GCE node
I20260806 19:47:16.040418 7973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:16.041512 7973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:16.042747 7973 hybrid_clock.cc:648] HybridClock initialized: now 1786045636042699 us; error 34 us; skew 500 ppm
I20260806 19:47:16.043123 7973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:47:16.046195 7973 webserver.cc:460] Webserver started at http://127.21.202.65:41087/ using document root <none> and password file <none>
I20260806 19:47:16.046985 7973 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:16.047077 7973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:16.047468 7973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:16.051024 7973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "6159e7e12af34b94aad9fab8929a995b"
format_stamp: "Formatted at 2026-08-06 19:47:16 on dist-test-slave-77v9"
server_key: "40dc5dd5987fa0fa3e57a7307bd4cb0d"
server_key_iv: "4ea1f3b35c49b1adf827bf94283f7b1d"
server_key_version: "encryptionkey@0"
I20260806 19:47:16.051903 7973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "6159e7e12af34b94aad9fab8929a995b"
format_stamp: "Formatted at 2026-08-06 19:47:16 on dist-test-slave-77v9"
server_key: "40dc5dd5987fa0fa3e57a7307bd4cb0d"
server_key_iv: "4ea1f3b35c49b1adf827bf94283f7b1d"
server_key_version: "encryptionkey@0"
I20260806 19:47:16.057758 7973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 19:47:16.062178 7988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:16.063994 7973 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260806 19:47:16.064172 7973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "6159e7e12af34b94aad9fab8929a995b"
format_stamp: "Formatted at 2026-08-06 19:47:16 on dist-test-slave-77v9"
server_key: "40dc5dd5987fa0fa3e57a7307bd4cb0d"
server_key_iv: "4ea1f3b35c49b1adf827bf94283f7b1d"
server_key_version: "encryptionkey@0"
I20260806 19:47:16.064388 7973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:16.093226 7973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:16.094136 7973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:16.094458 7973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:16.095593 7973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:16.097186 7973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:16.097277 7973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:16.097457 7973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:16.097558 7973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:16.136475 7973 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:39197
I20260806 19:47:16.136595 8100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:39197 every 8 connection(s)
I20260806 19:47:16.137987 7973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:47:16.147481 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 7973
I20260806 19:47:16.147918 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:47:16.149003 22313 external_mini_cluster.cc:1428] Setting key 6af677ffb2558ad0147d8d1a51fee127
I20260806 19:47:16.153237 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:44541
--builtin_ntp_servers=127.21.202.84:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 19:47:16.155580 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:16.141801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:37065 (local address 127.21.202.126:44541)
0806 19:47:16.142482 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:16.142491 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:16.144336 (+ 1845us) server_negotiation.cc:407] Connection header received
0806 19:47:16.146338 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:16.146343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:16.146443 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:16.146573 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:16.149346 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:16.150262 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:16.152201 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:16.152676 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:16.152802 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:16.154682 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:16.154688 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:16.154692 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:16.154714 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:16.154761 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:16.154775 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:16.154874 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:16.155200 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 19:47:16.155290 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":301,"threads_started":1}
I20260806 19:47:16.157253 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:16.142822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:44541 (local address 127.21.202.65:37065)
0806 19:47:16.144086 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:16.144133 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:16.145985 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:16.147240 (+ 1255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:16.147259 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:16.147457 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:16.148929 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:16.148976 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:16.150505 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:16.150511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:16.151955 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:16.151972 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:16.152988 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:16.153000 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:16.153340 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:16.154245 (+ 905us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:47:16.154251 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:16.154320 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:47:16.154444 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:16.154902 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:16.154917 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:16.154928 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 19:47:16.155143 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 19:47:16.155627 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":409,"threads_started":1}
I20260806 19:47:16.159081 8101 heartbeater.cc:347] Connected to a master server at 127.21.202.126:44541
I20260806 19:47:16.159494 8101 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:16.160334 8101 heartbeater.cc:511] Master 127.21.202.126:44541 requested a full tablet report, sending...
I20260806 19:47:16.162698 7914 ts_manager.cc:194] Registered new tserver with Master: 6159e7e12af34b94aad9fab8929a995b (127.21.202.65:39197)
I20260806 19:47:16.164336 7914 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.202.65:37065
W20260806 19:47:16.512326 8105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:16.512665 8105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:16.512760 8105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:16.520689 8105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:16.520923 8105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:16.521152 8105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:47:16.531476 8105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:44541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:16.532920 8105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:16.534711 8105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:16.548535 8110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:16.548905 8113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:16.548846 8111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:16.550032 8105 server_base.cc:1034] running on GCE node
I20260806 19:47:16.550712 8105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:16.551853 8105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:16.553122 8105 hybrid_clock.cc:648] HybridClock initialized: now 1786045636553055 us; error 57 us; skew 500 ppm
I20260806 19:47:16.553524 8105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:47:16.556078 8105 webserver.cc:460] Webserver started at http://127.21.202.66:36187/ using document root <none> and password file <none>
I20260806 19:47:16.556881 8105 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:16.557035 8105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:16.557405 8105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:16.560525 8105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "99c1dd1eee264535895cc5484bc077be"
format_stamp: "Formatted at 2026-08-06 19:47:16 on dist-test-slave-77v9"
server_key: "b75452a369327b1a92c00b953c947874"
server_key_iv: "337c5ac8218ea8995a287d9b04292ff9"
server_key_version: "encryptionkey@0"
I20260806 19:47:16.561444 8105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "99c1dd1eee264535895cc5484bc077be"
format_stamp: "Formatted at 2026-08-06 19:47:16 on dist-test-slave-77v9"
server_key: "b75452a369327b1a92c00b953c947874"
server_key_iv: "337c5ac8218ea8995a287d9b04292ff9"
server_key_version: "encryptionkey@0"
I20260806 19:47:16.567719 8105 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 19:47:16.572443 8119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:16.574275 8105 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:47:16.574520 8105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "99c1dd1eee264535895cc5484bc077be"
format_stamp: "Formatted at 2026-08-06 19:47:16 on dist-test-slave-77v9"
server_key: "b75452a369327b1a92c00b953c947874"
server_key_iv: "337c5ac8218ea8995a287d9b04292ff9"
server_key_version: "encryptionkey@0"
I20260806 19:47:16.574764 8105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:16.597975 8105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:16.598920 8105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:16.599262 8105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:16.600350 8105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:16.601979 8105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:16.602068 8105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:16.602192 8105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:16.602259 8105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:16.642936 8105 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:38637
I20260806 19:47:16.643152 8232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:38637 every 8 connection(s)
I20260806 19:47:16.644332 8105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:47:16.650679 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 8105
I20260806 19:47:16.650874 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:47:16.651507 22313 external_mini_cluster.cc:1428] Setting key 9d7e788943185130b8ea21bf16be525e
I20260806 19:47:16.655383 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:44541
--builtin_ntp_servers=127.21.202.84:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 19:47:16.660125 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:16.647304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:34251 (local address 127.21.202.126:44541)
0806 19:47:16.647549 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:16.647556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:16.649119 (+ 1563us) server_negotiation.cc:407] Connection header received
0806 19:47:16.651477 (+ 2358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:16.651482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:16.651589 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:16.651725 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:16.652983 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:16.654030 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:16.656933 (+ 2903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:16.657360 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:16.657470 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:16.659336 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:16.659341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:16.659344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:16.659365 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:16.659401 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:16.659413 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:16.659494 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:16.659776 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 19:47:16.659880 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 19:47:16.661904 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:16.647849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:44541 (local address 127.21.202.66:34251)
0806 19:47:16.648959 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:16.648998 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:16.651148 (+ 2150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:16.652025 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:16.652037 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:16.652153 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:16.652767 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:16.652784 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:16.654229 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:16.654235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:16.656665 (+ 2430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:16.656688 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:16.657647 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:16.657657 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:16.658046 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:16.658893 (+ 847us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:47:16.658900 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:16.658972 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:47:16.659101 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:16.659527 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:16.659541 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:16.659553 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 19:47:16.659776 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 19:47:16.660261 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":332,"threads_started":1}
I20260806 19:47:16.663661 8233 heartbeater.cc:347] Connected to a master server at 127.21.202.126:44541
I20260806 19:47:16.664094 8233 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:16.665176 8233 heartbeater.cc:511] Master 127.21.202.126:44541 requested a full tablet report, sending...
I20260806 19:47:16.666927 7914 ts_manager.cc:194] Registered new tserver with Master: 99c1dd1eee264535895cc5484bc077be (127.21.202.66:38637)
I20260806 19:47:16.667852 7914 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.202.66:34251
W20260806 19:47:17.026336 8236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:17.026686 8236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:17.026862 8236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:17.035178 8236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:17.035295 8236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:17.035480 8236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:47:17.045878 8236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:44541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:17.047314 8236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:17.049299 8236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:17.063621 8241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:17.063661 8244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:17.063599 8242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:17.065166 8236 server_base.cc:1034] running on GCE node
I20260806 19:47:17.065915 8236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:17.067027 8236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:17.068285 8236 hybrid_clock.cc:648] HybridClock initialized: now 1786045637068235 us; error 42 us; skew 500 ppm
I20260806 19:47:17.068909 8236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:47:17.071667 8236 webserver.cc:460] Webserver started at http://127.21.202.67:36837/ using document root <none> and password file <none>
I20260806 19:47:17.072438 8236 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:17.072576 8236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:17.073014 8236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:17.076188 8236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "156d86073d0842ab80c077d88ed7b9f1"
format_stamp: "Formatted at 2026-08-06 19:47:17 on dist-test-slave-77v9"
server_key: "58bd45e2073d7a89547df96757e690de"
server_key_iv: "f2de944e9e0796ca85d7a06f8e4aa584"
server_key_version: "encryptionkey@0"
I20260806 19:47:17.077105 8236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "156d86073d0842ab80c077d88ed7b9f1"
format_stamp: "Formatted at 2026-08-06 19:47:17 on dist-test-slave-77v9"
server_key: "58bd45e2073d7a89547df96757e690de"
server_key_iv: "f2de944e9e0796ca85d7a06f8e4aa584"
server_key_version: "encryptionkey@0"
I20260806 19:47:17.083192 8236 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 19:47:17.087039 8250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:17.088505 8236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:47:17.088714 8236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "156d86073d0842ab80c077d88ed7b9f1"
format_stamp: "Formatted at 2026-08-06 19:47:17 on dist-test-slave-77v9"
server_key: "58bd45e2073d7a89547df96757e690de"
server_key_iv: "f2de944e9e0796ca85d7a06f8e4aa584"
server_key_version: "encryptionkey@0"
I20260806 19:47:17.089052 8236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:17.103778 8236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:17.104713 8236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:17.105113 8236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:17.106200 8236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:17.107824 8236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:17.107961 8236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:17.108115 8236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:17.108212 8236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:17.147549 8236 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:42627
I20260806 19:47:17.147662 8363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:42627 every 8 connection(s)
I20260806 19:47:17.149011 8236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:47:17.152488 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 8236
I20260806 19:47:17.152673 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:47:17.153273 22313 external_mini_cluster.cc:1428] Setting key 72976fc82d1750a37e57d34d7dccbaf4
I20260806 19:47:17.163712 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.152038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:44543 (local address 127.21.202.126:44541)
0806 19:47:17.152324 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:17.152332 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:17.154206 (+ 1874us) server_negotiation.cc:407] Connection header received
0806 19:47:17.156499 (+ 2293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:17.156507 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:17.156609 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:17.156700 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:17.157986 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.158826 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.160455 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.161030 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.161143 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.162868 (+ 1725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:17.162882 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:17.162886 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:17.162909 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:17.162961 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:17.162976 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:17.163062 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:17.163345 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 19:47:17.163442 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260806 19:47:17.164870 8366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.152711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:44541 (local address 127.21.202.67:44543)
0806 19:47:17.153984 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:17.154066 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:17.156165 (+ 2099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:17.157018 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:17.157028 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:17.157124 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:17.157787 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:17.157805 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.159001 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.159005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:17.160294 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:17.160307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.161277 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.161283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:17.161523 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.162331 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:47:17.162335 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:17.162419 (+ 84us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:47:17.162531 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:17.163079 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:17.163088 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:17.163095 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 19:47:17.163262 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 19:47:17.163631 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":352,"threads_started":1}
I20260806 19:47:17.166504 8364 heartbeater.cc:347] Connected to a master server at 127.21.202.126:44541
I20260806 19:47:17.166858 8364 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:17.167800 8364 heartbeater.cc:511] Master 127.21.202.126:44541 requested a full tablet report, sending...
I20260806 19:47:17.168432 8101 heartbeater.cc:503] Master 127.21.202.126:44541 was elected leader, sending a full tablet report...
I20260806 19:47:17.169685 7912 ts_manager.cc:194] Registered new tserver with Master: 156d86073d0842ab80c077d88ed7b9f1 (127.21.202.67:42627)
I20260806 19:47:17.170737 7912 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.202.67:44543
I20260806 19:47:17.182837 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:47:17.194171 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.187590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55910 (local address 127.21.202.126:44541)
0806 19:47:17.187818 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:17.187823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:17.188263 (+ 440us) server_negotiation.cc:407] Connection header received
0806 19:47:17.188427 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:17.188431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:17.188501 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:17.188590 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:17.189213 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.190033 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.191464 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.191947 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.192051 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.193244 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:17.193249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:17.193252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:17.193273 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:17.193308 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:17.193321 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:17.193409 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:17.193690 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 19:47:17.193841 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 19:47:17.198577 7912 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:47:17.206439 7912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:47:17.232141 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.220788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:38637 (local address 127.0.0.1:37300)
0806 19:47:17.222254 (+ 1466us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:17.222342 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:17.222560 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:17.223849 (+ 1289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:17.223853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:17.223879 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:17.224391 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:17.224405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.226241 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.226250 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:17.228714 (+ 2464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:17.228729 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.230881 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.230888 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:17.231192 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.231257 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 19:47:17.231412 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 19:47:17.231635 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1291,"mutex_wait_us":119,"spinlock_wait_cycles":16000,"thread_start_us":1422,"threads_started":1}
I20260806 19:47:17.232394 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.221554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.21.202.66:38637)
0806 19:47:17.222078 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:17.222086 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:17.222653 (+ 567us) server_negotiation.cc:407] Connection header received
0806 19:47:17.222766 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:17.222776 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:17.223641 (+ 865us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:17.224394 (+ 753us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:17.224635 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.226101 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.228950 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.230626 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.230814 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.231108 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:17.231807 (+ 699us) server_negotiation.cc:299] Negotiation successful
0806 19:47:17.232005 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":883,"threads_started":1}
I20260806 19:47:17.233258 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.222550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:42627 (local address 127.0.0.1:33042)
0806 19:47:17.223085 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:17.223103 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:17.223214 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:17.225716 (+ 2502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:17.225724 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:17.225759 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:17.226115 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:17.226126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.228121 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.228127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:17.230109 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:17.230121 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.231903 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.231909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:17.232159 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.232200 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 19:47:17.232749 (+ 549us) client_negotiation.cc:240] Negotiation successful
0806 19:47:17.232914 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":19,"thread_start_us":748,"threads_started":1}
I20260806 19:47:17.233317 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.220197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55500 (local address 127.21.202.65:39197)
0806 19:47:17.221847 (+ 1650us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:17.221858 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:17.222776 (+ 918us) server_negotiation.cc:407] Connection header received
0806 19:47:17.222895 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:17.222902 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:17.223186 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:17.224105 (+ 919us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:17.224547 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.226314 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.229440 (+ 3126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.231011 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.231176 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.231669 (+ 493us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:17.232703 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0806 19:47:17.232940 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1418,"thread_start_us":1249,"threads_started":1}
I20260806 19:47:17.234743 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.223199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33042 (local address 127.21.202.67:42627)
0806 19:47:17.225064 (+ 1865us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:17.225074 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:17.225105 (+ 31us) server_negotiation.cc:407] Connection header received
0806 19:47:17.225197 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:17.225205 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:17.225420 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:17.225606 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:17.226340 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.227970 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.230306 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.231660 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.233902 (+ 2242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.234120 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:17.234173 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 19:47:17.234295 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1681,"thread_start_us":1487,"threads_started":1}
I20260806 19:47:17.235110 8376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.219833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:39197 (local address 127.0.0.1:55500)
0806 19:47:17.222254 (+ 2421us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:17.222362 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:17.222559 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:17.223466 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:17.223478 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:17.223528 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:17.224176 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:17.224204 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.226472 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.226480 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:17.229120 (+ 2640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:17.229135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.231428 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.231441 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:17.231640 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.231677 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:47:17.231786 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 19:47:17.231958 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2149,"mutex_wait_us":42,"spinlock_wait_cycles":4480,"thread_start_us":1237,"threads_started":1}
I20260806 19:47:17.237288 8168 tablet_service.cc:1467] Processing CreateTablet for tablet 3740689681314b71be25d602efc3e52a (DEFAULT_TABLE table=test-table [id=d4ef48c1999442ed84b79b7aacf4da42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:17.238827 8168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3740689681314b71be25d602efc3e52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:17.238484 8299 tablet_service.cc:1467] Processing CreateTablet for tablet 3740689681314b71be25d602efc3e52a (DEFAULT_TABLE table=test-table [id=d4ef48c1999442ed84b79b7aacf4da42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:17.238665 8036 tablet_service.cc:1467] Processing CreateTablet for tablet 3740689681314b71be25d602efc3e52a (DEFAULT_TABLE table=test-table [id=d4ef48c1999442ed84b79b7aacf4da42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:17.240017 8036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3740689681314b71be25d602efc3e52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:17.240016 8299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3740689681314b71be25d602efc3e52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:17.251209 8382 tablet_bootstrap.cc:492] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be: Bootstrap starting.
I20260806 19:47:17.255141 8383 tablet_bootstrap.cc:492] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1: Bootstrap starting.
I20260806 19:47:17.256299 8382 tablet_bootstrap.cc:654] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:17.257005 8384 tablet_bootstrap.cc:492] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b: Bootstrap starting.
I20260806 19:47:17.257911 8382 log.cc:824] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:17.259284 8383 tablet_bootstrap.cc:654] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:17.260432 8384 tablet_bootstrap.cc:654] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:17.260545 8383 log.cc:824] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:17.261716 8382 tablet_bootstrap.cc:492] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be: No bootstrap required, opened a new log
I20260806 19:47:17.261910 8384 log.cc:824] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:17.262080 8382 ts_tablet_manager.cc:1397] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260806 19:47:17.263911 8383 tablet_bootstrap.cc:492] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1: No bootstrap required, opened a new log
I20260806 19:47:17.264180 8383 ts_tablet_manager.cc:1397] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260806 19:47:17.265003 8384 tablet_bootstrap.cc:492] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b: No bootstrap required, opened a new log
I20260806 19:47:17.265265 8384 ts_tablet_manager.cc:1397] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260806 19:47:17.273300 8382 raft_consensus.cc:348] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6159e7e12af34b94aad9fab8929a995b" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39197 } } peers { permanent_uuid: "156d86073d0842ab80c077d88ed7b9f1" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42627 } } peers { permanent_uuid: "99c1dd1eee264535895cc5484bc077be" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38637 } }
I20260806 19:47:17.273273 8384 raft_consensus.cc:348] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6159e7e12af34b94aad9fab8929a995b" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39197 } } peers { permanent_uuid: "156d86073d0842ab80c077d88ed7b9f1" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42627 } } peers { permanent_uuid: "99c1dd1eee264535895cc5484bc077be" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38637 } }
I20260806 19:47:17.273792 8382 raft_consensus.cc:374] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:17.273804 8384 raft_consensus.cc:374] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:17.273901 8382 raft_consensus.cc:729] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99c1dd1eee264535895cc5484bc077be, State: Initialized, Role: FOLLOWER
I20260806 19:47:17.273904 8384 raft_consensus.cc:729] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6159e7e12af34b94aad9fab8929a995b, State: Initialized, Role: FOLLOWER
I20260806 19:47:17.274564 8384 consensus_queue.cc:260] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6159e7e12af34b94aad9fab8929a995b" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39197 } } peers { permanent_uuid: "156d86073d0842ab80c077d88ed7b9f1" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42627 } } peers { permanent_uuid: "99c1dd1eee264535895cc5484bc077be" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38637 } }
I20260806 19:47:17.274581 8382 consensus_queue.cc:260] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6159e7e12af34b94aad9fab8929a995b" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39197 } } peers { permanent_uuid: "156d86073d0842ab80c077d88ed7b9f1" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42627 } } peers { permanent_uuid: "99c1dd1eee264535895cc5484bc077be" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38637 } }
I20260806 19:47:17.275101 8383 raft_consensus.cc:348] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6159e7e12af34b94aad9fab8929a995b" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39197 } } peers { permanent_uuid: "156d86073d0842ab80c077d88ed7b9f1" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42627 } } peers { permanent_uuid: "99c1dd1eee264535895cc5484bc077be" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38637 } }
I20260806 19:47:17.275691 8383 raft_consensus.cc:374] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:17.275811 8383 raft_consensus.cc:729] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 156d86073d0842ab80c077d88ed7b9f1, State: Initialized, Role: FOLLOWER
I20260806 19:47:17.276716 8383 consensus_queue.cc:260] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6159e7e12af34b94aad9fab8929a995b" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39197 } } peers { permanent_uuid: "156d86073d0842ab80c077d88ed7b9f1" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42627 } } peers { permanent_uuid: "99c1dd1eee264535895cc5484bc077be" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38637 } }
I20260806 19:47:17.278234 8364 heartbeater.cc:503] Master 127.21.202.126:44541 was elected leader, sending a full tablet report...
I20260806 19:47:17.278954 8383 ts_tablet_manager.cc:1428] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260806 19:47:17.279976 8233 heartbeater.cc:503] Master 127.21.202.126:44541 was elected leader, sending a full tablet report...
I20260806 19:47:17.280771 8382 ts_tablet_manager.cc:1428] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be: Time spent starting tablet: real 0.019s user 0.014s sys 0.004s
I20260806 19:47:17.282357 8384 ts_tablet_manager.cc:1428] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b: Time spent starting tablet: real 0.017s user 0.007s sys 0.009s
I20260806 19:47:17.282407 8388 raft_consensus.cc:482] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:47:17.282694 8388 raft_consensus.cc:504] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6159e7e12af34b94aad9fab8929a995b" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39197 } } peers { permanent_uuid: "156d86073d0842ab80c077d88ed7b9f1" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42627 } } peers { permanent_uuid: "99c1dd1eee264535895cc5484bc077be" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38637 } }
I20260806 19:47:17.286080 8388 leader_election.cc:288] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 156d86073d0842ab80c077d88ed7b9f1 (127.21.202.67:42627), 99c1dd1eee264535895cc5484bc077be (127.21.202.66:38637)
I20260806 19:47:17.295743 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.285228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:57879 (local address 127.21.202.67:42627)
0806 19:47:17.285406 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:17.285413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:17.287165 (+ 1752us) server_negotiation.cc:407] Connection header received
0806 19:47:17.287480 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:17.287487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:17.287574 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:17.287643 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:17.288463 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.289488 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.292483 (+ 2995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.293815 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.294011 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.294080 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:17.295334 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0806 19:47:17.295479 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 19:47:17.295743 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.286722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:35945 (local address 127.21.202.66:38637)
0806 19:47:17.286924 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:17.286930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:17.287907 (+ 977us) server_negotiation.cc:407] Connection header received
0806 19:47:17.288127 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:17.288131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:17.288201 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:17.288278 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:17.288922 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.289895 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.292487 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.293818 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.294006 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.294080 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:17.295332 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0806 19:47:17.295479 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 19:47:17.296126 8391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.285615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:42627 (local address 127.21.202.65:57879)
0806 19:47:17.287059 (+ 1444us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:17.287079 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:17.287288 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:17.287879 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:17.287884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:17.287909 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:17.288343 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:17.288354 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.289621 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.289629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:17.292147 (+ 2518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:17.292168 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.294406 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.294428 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:17.294998 (+ 570us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.295137 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 19:47:17.295252 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 19:47:17.295469 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1300,"mutex_wait_us":215,"thread_start_us":1313,"threads_started":1}
I20260806 19:47:17.296253 8392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.286325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:38637 (local address 127.21.202.65:35945)
0806 19:47:17.287751 (+ 1426us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:17.287798 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:17.287979 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:17.288355 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:17.288359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:17.288385 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:17.288705 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:17.288713 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.290018 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.290023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:17.292251 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:17.292264 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.294413 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.294428 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:17.294996 (+ 568us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.295137 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 19:47:17.295252 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 19:47:17.295464 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1305,"mutex_wait_us":147,"spinlock_wait_cycles":21376,"thread_start_us":1071,"threads_started":1}
I20260806 19:47:17.297341 8188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3740689681314b71be25d602efc3e52a" candidate_uuid: "6159e7e12af34b94aad9fab8929a995b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99c1dd1eee264535895cc5484bc077be" is_pre_election: true
I20260806 19:47:17.297343 8319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3740689681314b71be25d602efc3e52a" candidate_uuid: "6159e7e12af34b94aad9fab8929a995b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "156d86073d0842ab80c077d88ed7b9f1" is_pre_election: true
I20260806 19:47:17.297886 8188 raft_consensus.cc:2454] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6159e7e12af34b94aad9fab8929a995b in term 0.
I20260806 19:47:17.297880 8319 raft_consensus.cc:2454] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6159e7e12af34b94aad9fab8929a995b in term 0.
I20260806 19:47:17.298650 7991 leader_election.cc:302] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 156d86073d0842ab80c077d88ed7b9f1, 6159e7e12af34b94aad9fab8929a995b, 99c1dd1eee264535895cc5484bc077be; no voters:
I20260806 19:47:17.299091 8388 raft_consensus.cc:2781] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:47:17.299261 8388 raft_consensus.cc:482] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:47:17.299379 8388 raft_consensus.cc:3037] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:17.301906 8388 raft_consensus.cc:504] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6159e7e12af34b94aad9fab8929a995b" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39197 } } peers { permanent_uuid: "156d86073d0842ab80c077d88ed7b9f1" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42627 } } peers { permanent_uuid: "99c1dd1eee264535895cc5484bc077be" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38637 } }
I20260806 19:47:17.302603 8388 leader_election.cc:288] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [CANDIDATE]: Term 1 election: Requested vote from peers 156d86073d0842ab80c077d88ed7b9f1 (127.21.202.67:42627), 99c1dd1eee264535895cc5484bc077be (127.21.202.66:38637)
I20260806 19:47:17.303135 8188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3740689681314b71be25d602efc3e52a" candidate_uuid: "6159e7e12af34b94aad9fab8929a995b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99c1dd1eee264535895cc5484bc077be"
I20260806 19:47:17.303139 8319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3740689681314b71be25d602efc3e52a" candidate_uuid: "6159e7e12af34b94aad9fab8929a995b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "156d86073d0842ab80c077d88ed7b9f1"
I20260806 19:47:17.303344 8188 raft_consensus.cc:3037] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:17.303372 8319 raft_consensus.cc:3037] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:17.306113 8188 raft_consensus.cc:2454] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6159e7e12af34b94aad9fab8929a995b in term 1.
I20260806 19:47:17.306284 8319 raft_consensus.cc:2454] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6159e7e12af34b94aad9fab8929a995b in term 1.
I20260806 19:47:17.306588 7991 leader_election.cc:302] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6159e7e12af34b94aad9fab8929a995b, 99c1dd1eee264535895cc5484bc077be; no voters:
I20260806 19:47:17.306905 8388 raft_consensus.cc:2781] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:17.307334 8388 raft_consensus.cc:686] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [term 1 LEADER]: Becoming Leader. State: Replica: 6159e7e12af34b94aad9fab8929a995b, State: Running, Role: LEADER
I20260806 19:47:17.307945 8388 consensus_queue.cc:237] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6159e7e12af34b94aad9fab8929a995b" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39197 } } peers { permanent_uuid: "156d86073d0842ab80c077d88ed7b9f1" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42627 } } peers { permanent_uuid: "99c1dd1eee264535895cc5484bc077be" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38637 } }
I20260806 19:47:17.312777 7912 catalog_manager.cc:5495] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6159e7e12af34b94aad9fab8929a995b (127.21.202.65). New cstate: current_term: 1 leader_uuid: "6159e7e12af34b94aad9fab8929a995b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6159e7e12af34b94aad9fab8929a995b" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "156d86073d0842ab80c077d88ed7b9f1" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99c1dd1eee264535895cc5484bc077be" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38637 } health_report { overall_health: UNKNOWN } } }
I20260806 19:47:17.339184 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.333251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55514 (local address 127.21.202.65:39197)
0806 19:47:17.333484 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:17.333489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:17.333508 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:47:17.333723 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:17.333727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:17.333832 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:17.333983 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:47:17.334452 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.335154 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.336694 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.337250 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.337368 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.337922 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:47:17.338463 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:47:17.338575 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:17.338763 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 19:47:17.338834 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 19:47:17.348892 8319 raft_consensus.cc:1261] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1 [term 1 FOLLOWER]: Refusing update from remote peer 6159e7e12af34b94aad9fab8929a995b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:47:17.348954 8188 raft_consensus.cc:1261] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be [term 1 FOLLOWER]: Refusing update from remote peer 6159e7e12af34b94aad9fab8929a995b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:47:17.350276 8393 consensus_queue.cc:1035] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [LEADER]: Connected to new peer: Peer: permanent_uuid: "156d86073d0842ab80c077d88ed7b9f1" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 42627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:47:17.350781 8388 consensus_queue.cc:1035] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [LEADER]: Connected to new peer: Peer: permanent_uuid: "99c1dd1eee264535895cc5484bc077be" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 38637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:47:17.364327 8397 mvcc.cc:205] Tried to move back new op lower bound from 7315642930568228864 to 7315642930417745920. Current Snapshot: MvccSnapshot[applied={T|T < 7315642930568228864}]
I20260806 19:47:17.382513 7912 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:47:17.382807 7912 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:47:17.386893 7912 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1cb32226d3049e0b263bf014ae5f9c1: Sending DeleteTablet for 3 replicas of tablet 3740689681314b71be25d602efc3e52a
I20260806 19:47:17.388370 8036 tablet_service.cc:1514] Processing DeleteTablet for tablet 3740689681314b71be25d602efc3e52a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:17 UTC) from {username='slave'} at 127.0.0.1:55500
I20260806 19:47:17.388370 8299 tablet_service.cc:1514] Processing DeleteTablet for tablet 3740689681314b71be25d602efc3e52a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:17 UTC) from {username='slave'} at 127.0.0.1:33042
I20260806 19:47:17.389528 8168 tablet_service.cc:1514] Processing DeleteTablet for tablet 3740689681314b71be25d602efc3e52a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:17 UTC) from {username='slave'} at 127.0.0.1:37300
I20260806 19:47:17.390100 8407 tablet_replica.cc:330] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b: stopping tablet replica
I20260806 19:47:17.390339 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 7973
I20260806 19:47:17.390724 8407 raft_consensus.cc:2229] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:47:17.391573 8406 tablet_replica.cc:330] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1: stopping tablet replica
I20260806 19:47:17.392355 8407 raft_consensus.cc:2258] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:17.393422 8406 raft_consensus.cc:2229] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:47:17.394204 8406 raft_consensus.cc:2258] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:17.395614 8407 ts_tablet_manager.cc:1905] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:17.398224 8406 ts_tablet_manager.cc:1905] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:17.403370 8407 ts_tablet_manager.cc:1918] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:17.403569 8407 log.cc:1196] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/3740689681314b71be25d602efc3e52a
I20260806 19:47:17.404174 8407 ts_tablet_manager.cc:1939] T 3740689681314b71be25d602efc3e52a P 6159e7e12af34b94aad9fab8929a995b: Deleting consensus metadata
I20260806 19:47:17.406229 7901 catalog_manager.cc:4841] TS 6159e7e12af34b94aad9fab8929a995b (127.21.202.65:39197): tablet 3740689681314b71be25d602efc3e52a (table test-table [id=d4ef48c1999442ed84b79b7aacf4da42]) successfully deleted
I20260806 19:47:17.411077 8408 tablet_replica.cc:330] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be: stopping tablet replica
I20260806 19:47:17.411566 8408 raft_consensus.cc:2229] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:47:17.412096 8408 raft_consensus.cc:2258] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:17.413342 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.392253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55530 (local address 127.21.202.65:39197)
0806 19:47:17.392523 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:17.392533 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:17.403756 (+ 11223us) server_negotiation.cc:407] Connection header received
0806 19:47:17.404404 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:17.404418 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:17.404575 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:17.404669 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:17.405507 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.406754 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.408911 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.409586 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.409737 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.411291 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:17.411306 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:17.411333 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:17.412178 (+ 845us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:17.412238 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:17.412265 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:17.412378 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:17.412832 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 19:47:17.412956 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 19:47:17.414170 8096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:17.415637 8406 ts_tablet_manager.cc:1918] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:17.415848 8406 log.cc:1196] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/3740689681314b71be25d602efc3e52a
I20260806 19:47:17.416097 8408 ts_tablet_manager.cc:1905] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:17.416478 8406 ts_tablet_manager.cc:1939] T 3740689681314b71be25d602efc3e52a P 156d86073d0842ab80c077d88ed7b9f1: Deleting consensus metadata
I20260806 19:47:17.418623 7901 catalog_manager.cc:4841] TS 156d86073d0842ab80c077d88ed7b9f1 (127.21.202.67:42627): tablet 3740689681314b71be25d602efc3e52a (table test-table [id=d4ef48c1999442ed84b79b7aacf4da42]) successfully deleted
I20260806 19:47:17.424095 8408 ts_tablet_manager.cc:1918] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:17.424288 8408 log.cc:1196] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/3740689681314b71be25d602efc3e52a
I20260806 19:47:17.424945 8408 ts_tablet_manager.cc:1939] T 3740689681314b71be25d602efc3e52a P 99c1dd1eee264535895cc5484bc077be: Deleting consensus metadata
I20260806 19:47:17.427215 7900 catalog_manager.cc:4841] TS 99c1dd1eee264535895cc5484bc077be (127.21.202.66:38637): tablet 3740689681314b71be25d602efc3e52a (table test-table [id=d4ef48c1999442ed84b79b7aacf4da42]) successfully deleted
I20260806 19:47:17.518603 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 7973
I20260806 19:47:17.540688 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 8105
I20260806 19:47:17.548269 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.541546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37304 (local address 127.21.202.66:38637)
0806 19:47:17.541761 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:17.541765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:17.541783 (+ 18us) server_negotiation.cc:407] Connection header received
0806 19:47:17.542004 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:17.542015 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:17.542128 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:17.542222 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:17.542593 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.543369 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.545017 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.545451 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.545560 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.546599 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:17.546608 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:17.546624 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:17.547277 (+ 653us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:17.547354 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:17.547387 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:17.547531 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:17.547719 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 19:47:17.547842 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 19:47:17.549068 8228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:17.639389 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 8105
I20260806 19:47:17.658623 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 8236
I20260806 19:47:17.666527 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:17.659517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.21.202.67:42627)
0806 19:47:17.659720 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:17.659728 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:17.659780 (+ 52us) server_negotiation.cc:407] Connection header received
0806 19:47:17.660021 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:17.660025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:17.660126 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:17.660241 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:17.660660 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.661612 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.663154 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:17.663580 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:17.663723 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:17.664900 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:17.664914 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:17.664940 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:17.665732 (+ 792us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:17.665788 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:17.665808 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:17.665935 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:17.666073 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 19:47:17.666188 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 19:47:17.667285 8359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:17.761907 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 8236
I20260806 19:47:17.782701 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 7883
I20260806 19:47:17.784176 7944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:17.897655 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 7883
2026-08-06T19:47:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2963 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:18 dist-test-slave-77v9 krb5kdc[8416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:18 dist-test-slave-77v9 krb5kdc[8416](info): set up 2 sockets
Aug 06 19:47:18 dist-test-slave-77v9 krb5kdc[8416](info): commencing operation
krb5kdc: starting...
W20260806 19:47:19.081439 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.006s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:19 dist-test-slave-77v9 krb5kdc[8416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:47:19.413041 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46403
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:35341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:46403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:19.782346 8429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:19.782747 8429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:19.782928 8429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:19.790768 8429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:19.790959 8429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:19.791069 8429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:19.791177 8429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:19.791253 8429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:19.802093 8429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:35341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:46403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:19.803593 8429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:19.805006 8429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:19.817801 8434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:19.817813 8435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:19.817866 8437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:19.818697 8429 server_base.cc:1034] running on GCE node
I20260806 19:47:19.819556 8429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:19.821640 8429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:19.822887 8429 hybrid_clock.cc:648] HybridClock initialized: now 1786045639822834 us; error 49 us; skew 500 ppm
I20260806 19:47:19.823261 8429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:19 dist-test-slave-77v9 krb5kdc[8416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045639, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:19.831401 8429 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:47:19.833242 8429 webserver.cc:460] Webserver started at http://127.21.202.126:36171/ using document root <none> and password file <none>
I20260806 19:47:19.834004 8429 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:19.834165 8429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:19.834491 8429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:19.837749 8429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "dcb990b936be4eda9ec73bbe0d5ef0da"
format_stamp: "Formatted at 2026-08-06 19:47:19 on dist-test-slave-77v9"
server_key: "25ccb6bcc0a13028de12d35eb5fd7f56"
server_key_iv: "db714a0a7a1f28b850116651896b6290"
server_key_version: "encryptionkey@0"
I20260806 19:47:19.838614 8429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "dcb990b936be4eda9ec73bbe0d5ef0da"
format_stamp: "Formatted at 2026-08-06 19:47:19 on dist-test-slave-77v9"
server_key: "25ccb6bcc0a13028de12d35eb5fd7f56"
server_key_iv: "db714a0a7a1f28b850116651896b6290"
server_key_version: "encryptionkey@0"
I20260806 19:47:19.844102 8429 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:47:19.848026 8444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:19.849790 8429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:47:19.850036 8429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "dcb990b936be4eda9ec73bbe0d5ef0da"
format_stamp: "Formatted at 2026-08-06 19:47:19 on dist-test-slave-77v9"
server_key: "25ccb6bcc0a13028de12d35eb5fd7f56"
server_key_iv: "db714a0a7a1f28b850116651896b6290"
server_key_version: "encryptionkey@0"
I20260806 19:47:19.850303 8429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:19.878625 8429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:19.884938 8429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:19.885298 8429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:19.907532 8429 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:46403
I20260806 19:47:19.907532 8495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:46403 every 8 connection(s)
I20260806 19:47:19.909361 8429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:19.910108 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 8429
I20260806 19:47:19.910336 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:19.910830 22313 external_mini_cluster.cc:1428] Setting key 0fe69c96ea8b1a02f438f9749fd7557c
I20260806 19:47:19.914762 8496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 19:47:19 dist-test-slave-77v9 krb5kdc[8416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:47:19.931614 8496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da: Bootstrap starting.
I20260806 19:47:19.935292 8496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:19.936547 8496 log.cc:824] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:19.939095 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:19.913019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39914 (local address 127.21.202.126:46403)
0806 19:47:19.913850 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:19.913862 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:19.914060 (+ 198us) server_negotiation.cc:407] Connection header received
0806 19:47:19.916278 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:19.916305 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:19.916708 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:19.917119 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:19.919467 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:19.921126 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:19.923565 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:19.924316 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:19.924462 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:19.930762 (+ 6300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:19.930775 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:19.930799 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:19.930851 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:19.934318 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:19.935307 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:19.935314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:19.935320 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:19.935555 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:19.936047 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:19.936052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:19.936054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:19.936732 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:19.937064 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:19.937653 (+ 589us) server_negotiation.cc:299] Negotiation successful
0806 19:47:19.938123 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":365,"threads_started":1}
I20260806 19:47:19.939839 8496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da: No bootstrap required, opened a new log
I20260806 19:47:19.949044 8496 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb990b936be4eda9ec73bbe0d5ef0da" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46403 } }
I20260806 19:47:19.949466 8496 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:19.949581 8496 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcb990b936be4eda9ec73bbe0d5ef0da, State: Initialized, Role: FOLLOWER
I20260806 19:47:19.950220 8496 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb990b936be4eda9ec73bbe0d5ef0da" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46403 } }
I20260806 19:47:19.950464 8496 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:19.950614 8496 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:19.950799 8496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:19.953554 8496 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb990b936be4eda9ec73bbe0d5ef0da" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46403 } }
I20260806 19:47:19.954100 8496 leader_election.cc:302] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcb990b936be4eda9ec73bbe0d5ef0da; no voters:
I20260806 19:47:19.954777 8496 leader_election.cc:288] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:19.954998 8501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:19.955862 8501 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [term 1 LEADER]: Becoming Leader. State: Replica: dcb990b936be4eda9ec73bbe0d5ef0da, State: Running, Role: LEADER
I20260806 19:47:19.956452 8501 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb990b936be4eda9ec73bbe0d5ef0da" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46403 } }
I20260806 19:47:19.957108 8496 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:19.959823 8502 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcb990b936be4eda9ec73bbe0d5ef0da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb990b936be4eda9ec73bbe0d5ef0da" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46403 } } }
I20260806 19:47:19.959940 8503 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcb990b936be4eda9ec73bbe0d5ef0da. Latest consensus state: current_term: 1 leader_uuid: "dcb990b936be4eda9ec73bbe0d5ef0da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb990b936be4eda9ec73bbe0d5ef0da" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46403 } } }
I20260806 19:47:19.960371 8502 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:19.960448 8503 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:19.963220 8508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:19.969376 8508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:19.979512 8508 catalog_manager.cc:1347] Generated new cluster ID: e5223c732fce48069da80b6aa8d6e2fd
I20260806 19:47:19.979655 8508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:19.994072 8508 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:19.995500 8508 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:20.005759 8508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da: Generated new TSK 0
I20260806 19:47:20.006714 8508 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:47:20.081113 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46403
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:35341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:20.440192 8522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:20.440676 8522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:20.440901 8522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:20.448377 8522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:20.448505 8522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:20.448622 8522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:47:20.460497 8522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:35341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:20.462314 8522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:20.463826 8522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:20.479735 8528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:20.479941 8527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:20.479804 8530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:20.481166 8522 server_base.cc:1034] running on GCE node
I20260806 19:47:20.481870 8522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:20.482976 8522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:20.484454 8522 hybrid_clock.cc:648] HybridClock initialized: now 1786045640484213 us; error 211 us; skew 500 ppm
I20260806 19:47:20.484879 8522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:20 dist-test-slave-77v9 krb5kdc[8416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045640, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:20.488298 8522 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:47:20.490041 8522 webserver.cc:460] Webserver started at http://127.21.202.65:45077/ using document root <none> and password file <none>
I20260806 19:47:20.490832 8522 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:20.491001 8522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:20.491356 8522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:20.494792 8522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "58e4cec5dc404cba872006f3b235263e"
format_stamp: "Formatted at 2026-08-06 19:47:20 on dist-test-slave-77v9"
server_key: "56741c37b7c9583fbd4f3e95bc1c04da"
server_key_iv: "fc8a1b123940b01ee1f6350662c76a26"
server_key_version: "encryptionkey@0"
I20260806 19:47:20.495657 8522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "58e4cec5dc404cba872006f3b235263e"
format_stamp: "Formatted at 2026-08-06 19:47:20 on dist-test-slave-77v9"
server_key: "56741c37b7c9583fbd4f3e95bc1c04da"
server_key_iv: "fc8a1b123940b01ee1f6350662c76a26"
server_key_version: "encryptionkey@0"
I20260806 19:47:20.501695 8522 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 19:47:20.505837 8537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:20.507460 8522 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:47:20.507661 8522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "58e4cec5dc404cba872006f3b235263e"
format_stamp: "Formatted at 2026-08-06 19:47:20 on dist-test-slave-77v9"
server_key: "56741c37b7c9583fbd4f3e95bc1c04da"
server_key_iv: "fc8a1b123940b01ee1f6350662c76a26"
server_key_version: "encryptionkey@0"
I20260806 19:47:20.507929 8522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:20.533272 8522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:20.537992 8522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:20.538333 8522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:20.539450 8522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:20.541040 8522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:20.541150 8522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:20.541308 8522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:20.541388 8522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:20.580978 8522 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:46485
I20260806 19:47:20.581081 8652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:46485 every 8 connection(s)
I20260806 19:47:20.582355 8522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:47:20.584769 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 8522
I20260806 19:47:20.585013 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:47:20.585698 22313 external_mini_cluster.cc:1428] Setting key 7c5e361d9de37215976514bf96362ef0
Aug 06 19:47:20 dist-test-slave-77v9 krb5kdc[8416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045640, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:47:20.604308 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:20.585363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:40723 (local address 127.21.202.126:46403)
0806 19:47:20.585881 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:20.585888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:20.587094 (+ 1206us) server_negotiation.cc:407] Connection header received
0806 19:47:20.589143 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:20.589149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:20.589235 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:20.589371 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:20.590919 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:20.591625 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:20.593259 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:20.593668 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:20.593805 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:20.597393 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:20.597398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:20.597402 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:20.597445 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:20.600158 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:20.601044 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:20.601053 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:20.601059 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:20.601219 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:20.601738 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:20.601746 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:20.601752 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:20.602164 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:20.602324 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:20.603514 (+ 1190us) server_negotiation.cc:299] Negotiation successful
0806 19:47:20.603792 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":251,"threads_started":1}
I20260806 19:47:20.605165 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:20.586024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46403 (local address 127.21.202.65:40723)
0806 19:47:20.586946 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:20.586986 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:20.588795 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:20.589627 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:20.589636 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:20.590174 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:20.590741 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:20.590758 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:20.591800 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:20.591807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:20.593110 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:20.593122 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:20.593907 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:20.593915 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:20.594169 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:20.594882 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:47:20.594906 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:20.597152 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:20.600305 (+ 3153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:20.600312 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:20.600325 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:20.600801 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:20.601365 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:20.601370 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:20.601372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:20.601598 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:20.602402 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:20.602457 (+ 55us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:20.603041 (+ 584us) client_negotiation.cc:769] Sending connection context
0806 19:47:20.603335 (+ 294us) client_negotiation.cc:240] Negotiation successful
0806 19:47:20.603828 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":360,"threads_started":1}
I20260806 19:47:20.607069 8653 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46403
I20260806 19:47:20.607400 8653 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:20.608162 8653 heartbeater.cc:511] Master 127.21.202.126:46403 requested a full tablet report, sending...
I20260806 19:47:20.610584 8461 ts_manager.cc:194] Registered new tserver with Master: 58e4cec5dc404cba872006f3b235263e (127.21.202.65:46485)
I20260806 19:47:20.612864 8461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:40723
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:47:20.662889 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46403
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:35341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:21.050437 8659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:21.051102 8659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:21.051316 8659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:21.061389 8659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:21.061584 8659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:21.061797 8659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:47:21.072604 8659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:35341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:21.075142 8659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:21.077477 8659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:21.097412 8664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:21.097468 8665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:21.097434 8667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:21.098441 8659 server_base.cc:1034] running on GCE node
I20260806 19:47:21.099115 8659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:21.100219 8659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:21.101493 8659 hybrid_clock.cc:648] HybridClock initialized: now 1786045641101439 us; error 48 us; skew 500 ppm
I20260806 19:47:21.101871 8659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:21 dist-test-slave-77v9 krb5kdc[8416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045641, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:21.105293 8659 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:47:21.107342 8659 webserver.cc:460] Webserver started at http://127.21.202.66:41277/ using document root <none> and password file <none>
I20260806 19:47:21.108196 8659 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:21.108327 8659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:21.108743 8659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:21.112119 8659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "c6d2b93a5f75405f835cb1ee404e464a"
format_stamp: "Formatted at 2026-08-06 19:47:21 on dist-test-slave-77v9"
server_key: "ca76ad6fc2e5eec353bd69e1b3261341"
server_key_iv: "2ee49df1e570c4ba3113aee49503793f"
server_key_version: "encryptionkey@0"
I20260806 19:47:21.113095 8659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "c6d2b93a5f75405f835cb1ee404e464a"
format_stamp: "Formatted at 2026-08-06 19:47:21 on dist-test-slave-77v9"
server_key: "ca76ad6fc2e5eec353bd69e1b3261341"
server_key_iv: "2ee49df1e570c4ba3113aee49503793f"
server_key_version: "encryptionkey@0"
I20260806 19:47:21.118904 8659 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:47:21.122748 8674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:21.124330 8659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 19:47:21.124549 8659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "c6d2b93a5f75405f835cb1ee404e464a"
format_stamp: "Formatted at 2026-08-06 19:47:21 on dist-test-slave-77v9"
server_key: "ca76ad6fc2e5eec353bd69e1b3261341"
server_key_iv: "2ee49df1e570c4ba3113aee49503793f"
server_key_version: "encryptionkey@0"
I20260806 19:47:21.124776 8659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:21.154058 8659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:21.160398 8659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:21.160773 8659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:21.162091 8659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:21.164696 8659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:21.164841 8659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:21.165010 8659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:21.165062 8659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:21.207075 8659 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:42153
I20260806 19:47:21.207213 8787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:42153 every 8 connection(s)
I20260806 19:47:21.208488 8659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:47:21.209408 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 8659
I20260806 19:47:21.209594 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:47:21.210248 22313 external_mini_cluster.cc:1428] Setting key e05c8745e8cfc4e9799743cb990c396b
Aug 06 19:47:21 dist-test-slave-77v9 krb5kdc[8416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045641, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:47:21.234544 8790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:21.211696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:47549 (local address 127.21.202.126:46403)
0806 19:47:21.212220 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:21.212226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:21.213463 (+ 1237us) server_negotiation.cc:407] Connection header received
0806 19:47:21.215385 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:21.215394 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:21.215487 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:21.215615 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:21.217528 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.218728 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.220429 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.221240 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.221400 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:21.225655 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:21.225663 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:21.225671 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:21.225721 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:21.229633 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:21.231195 (+ 1562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:21.231201 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:21.231203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:21.231478 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:21.232323 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:21.232327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:21.232329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:21.232695 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:21.232921 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:21.233990 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0806 19:47:21.234234 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":968,"threads_started":1}
I20260806 19:47:21.237144 8791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:21.212332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46403 (local address 127.21.202.66:47549)
0806 19:47:21.213290 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:21.213340 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:21.215098 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:21.215912 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:21.215923 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:21.216563 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:21.217305 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:21.217324 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.218925 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.218929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:21.220264 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:21.220278 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.221506 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.221512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:21.221774 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:21.222659 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:47:21.222684 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:21.225369 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:21.229904 (+ 4535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:21.229921 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:21.229952 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:21.230937 (+ 985us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:21.231729 (+ 792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:21.231738 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:21.231745 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:21.232151 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:21.232897 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:21.232908 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:21.233447 (+ 539us) client_negotiation.cc:769] Sending connection context
0806 19:47:21.233828 (+ 381us) client_negotiation.cc:240] Negotiation successful
0806 19:47:21.234858 (+ 1030us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":388,"threads_started":1}
I20260806 19:47:21.238898 8788 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46403
I20260806 19:47:21.239248 8788 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:21.240029 8788 heartbeater.cc:511] Master 127.21.202.126:46403 requested a full tablet report, sending...
I20260806 19:47:21.241729 8461 ts_manager.cc:194] Registered new tserver with Master: c6d2b93a5f75405f835cb1ee404e464a (127.21.202.66:42153)
I20260806 19:47:21.242637 8461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:47549
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:47:21.294090 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46403
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:35341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:47:21.616918 8653 heartbeater.cc:503] Master 127.21.202.126:46403 was elected leader, sending a full tablet report...
W20260806 19:47:21.674795 8794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:21.675163 8794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:21.675364 8794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:21.683314 8794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:21.683501 8794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:21.683722 8794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:47:21.694205 8794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:35341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:46403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:21.695660 8794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:21.697063 8794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:21.711836 8799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:21.711918 8800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:21.711884 8802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:21.712443 8794 server_base.cc:1034] running on GCE node
I20260806 19:47:21.713382 8794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:21.714448 8794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:21.715714 8794 hybrid_clock.cc:648] HybridClock initialized: now 1786045641715644 us; error 56 us; skew 500 ppm
I20260806 19:47:21.716182 8794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:21 dist-test-slave-77v9 krb5kdc[8416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045641, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:21.719506 8794 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:47:21.721104 8794 webserver.cc:460] Webserver started at http://127.21.202.67:46659/ using document root <none> and password file <none>
I20260806 19:47:21.721799 8794 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:21.721904 8794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:21.722174 8794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:21.725255 8794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "734a0a001b714110b795c8a2e8adbe6c"
format_stamp: "Formatted at 2026-08-06 19:47:21 on dist-test-slave-77v9"
server_key: "8560ca39407697028f402c7e9ac7fccb"
server_key_iv: "248759eb7de749e2319ac61493c69fe1"
server_key_version: "encryptionkey@0"
I20260806 19:47:21.726039 8794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "734a0a001b714110b795c8a2e8adbe6c"
format_stamp: "Formatted at 2026-08-06 19:47:21 on dist-test-slave-77v9"
server_key: "8560ca39407697028f402c7e9ac7fccb"
server_key_iv: "248759eb7de749e2319ac61493c69fe1"
server_key_version: "encryptionkey@0"
I20260806 19:47:21.731745 8794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:47:21.735505 8809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:21.737421 8794 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 19:47:21.737601 8794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "734a0a001b714110b795c8a2e8adbe6c"
format_stamp: "Formatted at 2026-08-06 19:47:21 on dist-test-slave-77v9"
server_key: "8560ca39407697028f402c7e9ac7fccb"
server_key_iv: "248759eb7de749e2319ac61493c69fe1"
server_key_version: "encryptionkey@0"
I20260806 19:47:21.737850 8794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:21.756546 8794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:21.761509 8794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:21.761834 8794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:21.762972 8794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:21.764528 8794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:21.764603 8794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:21.764748 8794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:21.764798 8794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:21.799484 8794 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:35419
I20260806 19:47:21.799533 8922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:35419 every 8 connection(s)
I20260806 19:47:21.800920 8794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:47:21.809101 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 8794
I20260806 19:47:21.809311 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:47:21.809900 22313 external_mini_cluster.cc:1428] Setting key af4ae0136a5cbd28a56a0654b0edd6e1
Aug 06 19:47:21 dist-test-slave-77v9 krb5kdc[8416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045641, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:47:21.826894 8925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:21.804470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:52231 (local address 127.21.202.126:46403)
0806 19:47:21.805021 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:21.805026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:21.805947 (+ 921us) server_negotiation.cc:407] Connection header received
0806 19:47:21.807883 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:21.807888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:21.807994 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:21.808118 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:21.810253 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.811286 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.813530 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.813945 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.814058 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:21.819159 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:21.819165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:21.819168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:21.819207 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:21.822775 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:21.823756 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:21.823763 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:21.823767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:21.823930 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:21.824524 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:21.824529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:21.824532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:21.825004 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:21.825175 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:21.826299 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0806 19:47:21.826537 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":273,"threads_started":1}
I20260806 19:47:21.828341 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:21.805002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46403 (local address 127.21.202.67:52231)
0806 19:47:21.805755 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:21.805801 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:21.807603 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:21.808371 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:21.808379 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:21.809180 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:21.810042 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:21.810067 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.811509 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.811515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:21.813355 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:21.813375 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.814498 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.814514 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:21.815376 (+ 862us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:21.816239 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:47:21.816272 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:21.818673 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:21.822998 (+ 4325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:21.823006 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:21.823026 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:21.823573 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:21.824088 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:21.824093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:21.824095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:21.824363 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:21.825193 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:21.825202 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:21.825658 (+ 456us) client_negotiation.cc:769] Sending connection context
0806 19:47:21.826076 (+ 418us) client_negotiation.cc:240] Negotiation successful
0806 19:47:21.827132 (+ 1056us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":316,"threads_started":1}
I20260806 19:47:21.830230 8923 heartbeater.cc:347] Connected to a master server at 127.21.202.126:46403
I20260806 19:47:21.830626 8923 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:21.831473 8923 heartbeater.cc:511] Master 127.21.202.126:46403 requested a full tablet report, sending...
I20260806 19:47:21.833282 8461 ts_manager.cc:194] Registered new tserver with Master: 734a0a001b714110b795c8a2e8adbe6c (127.21.202.67:35419)
I20260806 19:47:21.834182 8461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:52231
I20260806 19:47:21.839998 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:47:21.860442 8925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:21.844966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.21.202.126:46403)
0806 19:47:21.845153 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:21.845158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:21.845470 (+ 312us) server_negotiation.cc:407] Connection header received
0806 19:47:21.845765 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:21.845771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:21.845878 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:21.846022 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:21.847595 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.848669 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.850455 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.851095 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.851210 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:21.854144 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:21.854153 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:21.854160 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:21.854212 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:21.857294 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:21.858388 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:21.858393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:21.858395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:21.858523 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:21.858990 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:21.858993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:21.858996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:21.859365 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:21.859584 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:21.860047 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 19:47:21.860195 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 19:47:21.865329 8461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:47:21.872547 8461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:47:21.900981 8936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:21.885426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:46485 (local address 127.0.0.1:50758)
0806 19:47:21.886216 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:21.886257 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:21.886451 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:21.887189 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:21.887196 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:21.887236 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:21.887759 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:21.887780 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.891820 (+ 4040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.891826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:21.896052 (+ 4226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:21.896071 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.899850 (+ 3779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.899857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:21.900125 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:21.900185 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 19:47:21.900334 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 19:47:21.900554 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":911,"threads_started":1}
I20260806 19:47:21.901073 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:21.887463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52032 (local address 127.21.202.66:42153)
0806 19:47:21.888148 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:21.888156 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:21.888662 (+ 506us) server_negotiation.cc:407] Connection header received
0806 19:47:21.888766 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:21.888773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:21.889073 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:21.889273 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:21.890222 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.891723 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.898190 (+ 6467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.899612 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.899754 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:21.899957 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:21.900517 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 19:47:21.900707 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":329,"threads_started":1}
I20260806 19:47:21.901793 8938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:21.887175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:42153 (local address 127.0.0.1:52032)
0806 19:47:21.888397 (+ 1222us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:21.888418 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:21.888543 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:21.889540 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:21.889544 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:21.889568 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:21.890026 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:21.890041 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.895808 (+ 5767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.895814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:21.897971 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:21.897986 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.900112 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.900118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:21.900292 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:21.900334 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 19:47:21.900406 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 19:47:21.901370 (+ 964us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1096,"mutex_wait_us":807,"thread_start_us":936,"threads_started":1}
I20260806 19:47:21.902967 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:21.885764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.21.202.65:46485)
0806 19:47:21.886452 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:21.886483 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:21.886516 (+ 33us) server_negotiation.cc:407] Connection header received
0806 19:47:21.886707 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:21.886715 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:21.886957 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:21.887176 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:21.888017 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.891204 (+ 3187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.896297 (+ 5093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.899261 (+ 2964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.899429 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:21.899691 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:21.901610 (+ 1919us) server_negotiation.cc:299] Negotiation successful
0806 19:47:21.901806 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":360,"threads_started":1}
I20260806 19:47:21.906733 8722 tablet_service.cc:1467] Processing CreateTablet for tablet 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 (DEFAULT_TABLE table=test-table [id=2d035e486af14b90aef1eabdcb20ff90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:21.906736 8586 tablet_service.cc:1467] Processing CreateTablet for tablet 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 (DEFAULT_TABLE table=test-table [id=2d035e486af14b90aef1eabdcb20ff90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:21.908322 8586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a36ed26a4ba4ebc8f19e1a0ba5cdf39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:21.909754 8936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:21.888477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:35419 (local address 127.0.0.1:59260)
0806 19:47:21.901304 (+ 12827us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:21.901324 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:21.901430 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:21.902583 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:21.902588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:21.902622 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:21.902847 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:21.902860 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.904837 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.904843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:21.906172 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:21.906186 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.908466 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.908473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:21.908677 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:21.908713 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 19:47:21.908795 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 19:47:21.909177 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12703,"thread_start_us":13922,"threads_started":1}
I20260806 19:47:21.910181 8722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a36ed26a4ba4ebc8f19e1a0ba5cdf39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:21.910702 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:21.888920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59260 (local address 127.21.202.67:35419)
0806 19:47:21.901435 (+ 12515us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:21.901448 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:21.901537 (+ 89us) server_negotiation.cc:407] Connection header received
0806 19:47:21.901670 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:21.901683 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:21.902003 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:21.902265 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:21.903044 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.904619 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.906339 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:21.907947 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:21.908077 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:21.910043 (+ 1966us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:21.910140 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 19:47:21.910370 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12305,"thread_start_us":13096,"threads_started":1}
I20260806 19:47:21.915359 8857 tablet_service.cc:1467] Processing CreateTablet for tablet 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 (DEFAULT_TABLE table=test-table [id=2d035e486af14b90aef1eabdcb20ff90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:21.917295 8857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a36ed26a4ba4ebc8f19e1a0ba5cdf39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:21.933748 8942 tablet_bootstrap.cc:492] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e: Bootstrap starting.
I20260806 19:47:21.936224 8944 tablet_bootstrap.cc:492] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c: Bootstrap starting.
I20260806 19:47:21.936229 8943 tablet_bootstrap.cc:492] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a: Bootstrap starting.
I20260806 19:47:21.937908 8942 tablet_bootstrap.cc:654] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:21.939352 8942 log.cc:824] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:21.940424 8944 tablet_bootstrap.cc:654] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:21.940783 8943 tablet_bootstrap.cc:654] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:21.941718 8944 log.cc:824] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:21.942289 8943 log.cc:824] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:21.943354 8942 tablet_bootstrap.cc:492] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e: No bootstrap required, opened a new log
I20260806 19:47:21.943784 8942 ts_tablet_manager.cc:1397] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260806 19:47:21.945567 8943 tablet_bootstrap.cc:492] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a: No bootstrap required, opened a new log
I20260806 19:47:21.945757 8944 tablet_bootstrap.cc:492] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c: No bootstrap required, opened a new log
I20260806 19:47:21.945885 8943 ts_tablet_manager.cc:1397] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260806 19:47:21.946030 8944 ts_tablet_manager.cc:1397] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 19:47:21.953894 8943 raft_consensus.cc:348] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e4cec5dc404cba872006f3b235263e" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 46485 } } peers { permanent_uuid: "c6d2b93a5f75405f835cb1ee404e464a" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42153 } } peers { permanent_uuid: "734a0a001b714110b795c8a2e8adbe6c" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35419 } }
I20260806 19:47:21.954330 8943 raft_consensus.cc:374] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:21.954425 8943 raft_consensus.cc:729] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6d2b93a5f75405f835cb1ee404e464a, State: Initialized, Role: FOLLOWER
I20260806 19:47:21.955237 8943 consensus_queue.cc:260] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e4cec5dc404cba872006f3b235263e" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 46485 } } peers { permanent_uuid: "c6d2b93a5f75405f835cb1ee404e464a" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42153 } } peers { permanent_uuid: "734a0a001b714110b795c8a2e8adbe6c" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35419 } }
I20260806 19:47:21.955557 8942 raft_consensus.cc:348] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e4cec5dc404cba872006f3b235263e" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 46485 } } peers { permanent_uuid: "c6d2b93a5f75405f835cb1ee404e464a" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42153 } } peers { permanent_uuid: "734a0a001b714110b795c8a2e8adbe6c" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35419 } }
I20260806 19:47:21.956116 8942 raft_consensus.cc:374] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:21.956233 8942 raft_consensus.cc:729] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58e4cec5dc404cba872006f3b235263e, State: Initialized, Role: FOLLOWER
I20260806 19:47:21.956667 8788 heartbeater.cc:503] Master 127.21.202.126:46403 was elected leader, sending a full tablet report...
I20260806 19:47:21.957247 8943 ts_tablet_manager.cc:1428] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260806 19:47:21.957087 8942 consensus_queue.cc:260] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e4cec5dc404cba872006f3b235263e" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 46485 } } peers { permanent_uuid: "c6d2b93a5f75405f835cb1ee404e464a" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42153 } } peers { permanent_uuid: "734a0a001b714110b795c8a2e8adbe6c" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35419 } }
I20260806 19:47:21.959012 8942 ts_tablet_manager.cc:1428] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260806 19:47:21.959290 8944 raft_consensus.cc:348] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e4cec5dc404cba872006f3b235263e" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 46485 } } peers { permanent_uuid: "c6d2b93a5f75405f835cb1ee404e464a" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42153 } } peers { permanent_uuid: "734a0a001b714110b795c8a2e8adbe6c" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35419 } }
I20260806 19:47:21.959803 8944 raft_consensus.cc:374] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:21.959928 8944 raft_consensus.cc:729] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 734a0a001b714110b795c8a2e8adbe6c, State: Initialized, Role: FOLLOWER
W20260806 19:47:21.960640 8789 tablet.cc:2369] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:47:21.960547 8944 consensus_queue.cc:260] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e4cec5dc404cba872006f3b235263e" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 46485 } } peers { permanent_uuid: "c6d2b93a5f75405f835cb1ee404e464a" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42153 } } peers { permanent_uuid: "734a0a001b714110b795c8a2e8adbe6c" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35419 } }
I20260806 19:47:21.967864 8923 heartbeater.cc:503] Master 127.21.202.126:46403 was elected leader, sending a full tablet report...
I20260806 19:47:21.969256 8944 ts_tablet_manager.cc:1428] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c: Time spent starting tablet: real 0.023s user 0.017s sys 0.004s
W20260806 19:47:22.053097 8924 tablet.cc:2369] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:47:22.085253 8654 tablet.cc:2369] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:47:22.176110 8950 raft_consensus.cc:482] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:47:22.176549 8950 raft_consensus.cc:504] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e4cec5dc404cba872006f3b235263e" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 46485 } } peers { permanent_uuid: "c6d2b93a5f75405f835cb1ee404e464a" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42153 } } peers { permanent_uuid: "734a0a001b714110b795c8a2e8adbe6c" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35419 } }
I20260806 19:47:22.179062 8950 leader_election.cc:288] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58e4cec5dc404cba872006f3b235263e (127.21.202.65:46485), c6d2b93a5f75405f835cb1ee404e464a (127.21.202.66:42153)
I20260806 19:47:22.186735 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:22.179560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:46485 (local address 127.21.202.67:35245)
0806 19:47:22.179817 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:22.179842 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:22.180009 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:22.180702 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:22.180709 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:22.180749 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:22.181043 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:22.181058 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.182711 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.182716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:22.184409 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:22.184424 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.185912 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.185919 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:22.186132 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:22.186169 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:47:22.186272 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 19:47:22.186418 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 19:47:22.186883 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:22.179475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:35245 (local address 127.21.202.65:46485)
0806 19:47:22.179731 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:22.179738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:22.180164 (+ 426us) server_negotiation.cc:407] Connection header received
0806 19:47:22.180268 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:22.180277 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:22.180377 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:22.180494 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:22.181241 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.182555 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.184607 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.185735 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.185928 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:22.186007 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:22.186329 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 19:47:22.186556 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 19:47:22.187913 8607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a36ed26a4ba4ebc8f19e1a0ba5cdf39" candidate_uuid: "734a0a001b714110b795c8a2e8adbe6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58e4cec5dc404cba872006f3b235263e" is_pre_election: true
I20260806 19:47:22.188503 8607 raft_consensus.cc:2454] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 734a0a001b714110b795c8a2e8adbe6c in term 0.
I20260806 19:47:22.188617 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:22.180170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:53513 (local address 127.21.202.66:42153)
0806 19:47:22.180426 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:22.180435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:22.181098 (+ 663us) server_negotiation.cc:407] Connection header received
0806 19:47:22.181301 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:22.181306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:22.181377 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:22.181476 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:22.182083 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.182912 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.186161 (+ 3249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.187290 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.187474 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:22.187555 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:22.188215 (+ 660us) server_negotiation.cc:299] Negotiation successful
0806 19:47:22.188356 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 19:47:22.188938 8951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:22.180006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:42153 (local address 127.21.202.67:53513)
0806 19:47:22.180978 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:22.180993 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:22.181151 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:22.181492 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:22.181496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:22.181522 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:22.181920 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:22.181929 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.183100 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.183109 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:22.185920 (+ 2811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:22.185937 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.187749 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.187758 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:22.187998 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:22.188049 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 19:47:22.188157 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 19:47:22.188365 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"thread_start_us":634,"threads_started":1}
I20260806 19:47:22.189306 8813 leader_election.cc:302] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58e4cec5dc404cba872006f3b235263e, 734a0a001b714110b795c8a2e8adbe6c; no voters:
I20260806 19:47:22.189782 8950 raft_consensus.cc:2781] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:47:22.189962 8950 raft_consensus.cc:482] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:47:22.190084 8950 raft_consensus.cc:3037] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:22.190546 8742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a36ed26a4ba4ebc8f19e1a0ba5cdf39" candidate_uuid: "734a0a001b714110b795c8a2e8adbe6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6d2b93a5f75405f835cb1ee404e464a" is_pre_election: true
I20260806 19:47:22.191018 8742 raft_consensus.cc:2454] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 734a0a001b714110b795c8a2e8adbe6c in term 0.
I20260806 19:47:22.194159 8950 raft_consensus.cc:504] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e4cec5dc404cba872006f3b235263e" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 46485 } } peers { permanent_uuid: "c6d2b93a5f75405f835cb1ee404e464a" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42153 } } peers { permanent_uuid: "734a0a001b714110b795c8a2e8adbe6c" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35419 } }
I20260806 19:47:22.195055 8950 leader_election.cc:288] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [CANDIDATE]: Term 1 election: Requested vote from peers 58e4cec5dc404cba872006f3b235263e (127.21.202.65:46485), c6d2b93a5f75405f835cb1ee404e464a (127.21.202.66:42153)
I20260806 19:47:22.195573 8742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a36ed26a4ba4ebc8f19e1a0ba5cdf39" candidate_uuid: "734a0a001b714110b795c8a2e8adbe6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6d2b93a5f75405f835cb1ee404e464a"
I20260806 19:47:22.195633 8607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a36ed26a4ba4ebc8f19e1a0ba5cdf39" candidate_uuid: "734a0a001b714110b795c8a2e8adbe6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58e4cec5dc404cba872006f3b235263e"
I20260806 19:47:22.195796 8742 raft_consensus.cc:3037] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:22.195899 8607 raft_consensus.cc:3037] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:22.198158 8742 raft_consensus.cc:2454] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 734a0a001b714110b795c8a2e8adbe6c in term 1.
I20260806 19:47:22.198621 8813 leader_election.cc:302] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 734a0a001b714110b795c8a2e8adbe6c, c6d2b93a5f75405f835cb1ee404e464a; no voters:
I20260806 19:47:22.198932 8950 raft_consensus.cc:2781] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:22.199529 8950 raft_consensus.cc:686] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [term 1 LEADER]: Becoming Leader. State: Replica: 734a0a001b714110b795c8a2e8adbe6c, State: Running, Role: LEADER
I20260806 19:47:22.199992 8607 raft_consensus.cc:2454] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 734a0a001b714110b795c8a2e8adbe6c in term 1.
I20260806 19:47:22.200096 8950 consensus_queue.cc:237] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e4cec5dc404cba872006f3b235263e" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 46485 } } peers { permanent_uuid: "c6d2b93a5f75405f835cb1ee404e464a" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42153 } } peers { permanent_uuid: "734a0a001b714110b795c8a2e8adbe6c" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35419 } }
I20260806 19:47:22.205264 8461 catalog_manager.cc:5495] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 734a0a001b714110b795c8a2e8adbe6c (127.21.202.67). New cstate: current_term: 1 leader_uuid: "734a0a001b714110b795c8a2e8adbe6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e4cec5dc404cba872006f3b235263e" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6d2b93a5f75405f835cb1ee404e464a" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "734a0a001b714110b795c8a2e8adbe6c" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 35419 } health_report { overall_health: HEALTHY } } }
I20260806 19:47:22.270071 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:22.263768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59274 (local address 127.21.202.67:35419)
0806 19:47:22.263970 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:22.263976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:22.264227 (+ 251us) server_negotiation.cc:407] Connection header received
0806 19:47:22.264481 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:22.264485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:22.264574 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:22.264731 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:47:22.265169 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.265919 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.267488 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.268052 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.268190 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:22.268803 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:47:22.269410 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:47:22.269601 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:22.269768 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 19:47:22.269818 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 19:47:22.278609 8742 raft_consensus.cc:1261] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a [term 1 FOLLOWER]: Refusing update from remote peer 734a0a001b714110b795c8a2e8adbe6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:47:22.278903 8607 raft_consensus.cc:1261] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e [term 1 FOLLOWER]: Refusing update from remote peer 734a0a001b714110b795c8a2e8adbe6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:47:22.279732 8950 consensus_queue.cc:1035] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6d2b93a5f75405f835cb1ee404e464a" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:47:22.280160 8952 consensus_queue.cc:1035] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "58e4cec5dc404cba872006f3b235263e" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:47:22.294817 8956 mvcc.cc:205] Tried to move back new op lower bound from 7315642950762278912 to 7315642950457012224. Current Snapshot: MvccSnapshot[applied={T|T < 7315642950762278912}]
I20260806 19:47:22.304836 8957 mvcc.cc:205] Tried to move back new op lower bound from 7315642950762278912 to 7315642950457012224. Current Snapshot: MvccSnapshot[applied={T|T < 7315642950762278912}]
I20260806 19:47:22.316490 8461 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:47:22.316783 8461 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:47:22.320883 8461 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dcb990b936be4eda9ec73bbe0d5ef0da: Sending DeleteTablet for 3 replicas of tablet 1a36ed26a4ba4ebc8f19e1a0ba5cdf39
I20260806 19:47:22.322175 8586 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:22 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:50758
I20260806 19:47:22.322381 8857 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:22 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:59260
I20260806 19:47:22.322386 8722 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:22 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:52032
I20260806 19:47:22.328848 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 8522
I20260806 19:47:22.335527 8965 tablet_replica.cc:330] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a: stopping tablet replica
I20260806 19:47:22.336108 8965 raft_consensus.cc:2229] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:47:22.336673 8965 raft_consensus.cc:2258] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:22.339403 8965 ts_tablet_manager.cc:1905] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:22.341491 8966 tablet_replica.cc:330] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c: stopping tablet replica
I20260806 19:47:22.341969 8966 raft_consensus.cc:2229] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:47:22.342371 8964 tablet_replica.cc:330] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e: stopping tablet replica
I20260806 19:47:22.342767 8964 raft_consensus.cc:2229] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:47:22.342778 8966 raft_consensus.cc:2258] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:22.343276 8964 raft_consensus.cc:2258] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:22.345161 8966 ts_tablet_manager.cc:1905] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:22.345386 8964 ts_tablet_manager.cc:1905] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:22.352756 8964 ts_tablet_manager.cc:1918] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:22.352989 8964 log.cc:1196] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/1a36ed26a4ba4ebc8f19e1a0ba5cdf39
Aug 06 19:47:22 dist-test-slave-77v9 krb5kdc[8416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:47:22.353277 8966 ts_tablet_manager.cc:1918] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:22.353467 8966 log.cc:1196] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/1a36ed26a4ba4ebc8f19e1a0ba5cdf39
I20260806 19:47:22.353557 8964 ts_tablet_manager.cc:1939] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 58e4cec5dc404cba872006f3b235263e: Deleting consensus metadata
I20260806 19:47:22.354033 8966 ts_tablet_manager.cc:1939] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P 734a0a001b714110b795c8a2e8adbe6c: Deleting consensus metadata
I20260806 19:47:22.356029 8448 catalog_manager.cc:4841] TS 734a0a001b714110b795c8a2e8adbe6c (127.21.202.67:35419): tablet 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 (table test-table [id=2d035e486af14b90aef1eabdcb20ff90]) successfully deleted
I20260806 19:47:22.356455 8448 catalog_manager.cc:4841] TS 58e4cec5dc404cba872006f3b235263e (127.21.202.65:46485): tablet 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 (table test-table [id=2d035e486af14b90aef1eabdcb20ff90]) successfully deleted
I20260806 19:47:22.361894 8965 ts_tablet_manager.cc:1918] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:22.362084 8965 log.cc:1196] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/1a36ed26a4ba4ebc8f19e1a0ba5cdf39
I20260806 19:47:22.362656 8965 ts_tablet_manager.cc:1939] T 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 P c6d2b93a5f75405f835cb1ee404e464a: Deleting consensus metadata
I20260806 19:47:22.363312 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:22.330481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50760 (local address 127.21.202.65:46485)
0806 19:47:22.330981 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:22.330991 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:22.341659 (+ 10668us) server_negotiation.cc:407] Connection header received
0806 19:47:22.341875 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:22.341881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:22.342005 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:22.342154 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:22.344593 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.345722 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.347839 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.348470 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.348569 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:22.355191 (+ 6622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:22.355208 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:22.355227 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:22.355281 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:22.359360 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:22.360114 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:22.360129 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:22.360132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:22.360289 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:22.360942 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:22.360947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:22.360953 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:22.361344 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:22.361585 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:22.362402 (+ 817us) server_negotiation.cc:299] Negotiation successful
0806 19:47:22.362919 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260806 19:47:22.364089 8647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:22.364584 8448 catalog_manager.cc:4841] TS c6d2b93a5f75405f835cb1ee404e464a (127.21.202.66:42153): tablet 1a36ed26a4ba4ebc8f19e1a0ba5cdf39 (table test-table [id=2d035e486af14b90aef1eabdcb20ff90]) successfully deleted
I20260806 19:47:22.479480 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 8522
I20260806 19:47:22.506394 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 8659
Aug 06 19:47:22 dist-test-slave-77v9 krb5kdc[8416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:47:22.526419 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:22.507322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52034 (local address 127.21.202.66:42153)
0806 19:47:22.507507 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:22.507511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:22.507530 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:47:22.507852 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:22.507857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:22.507926 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:22.508040 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:22.509802 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.510531 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.512473 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.513107 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.513214 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:22.518906 (+ 5692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:22.518927 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:22.518952 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:22.519002 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:22.522930 (+ 3928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:22.523884 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:22.523894 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:22.523898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:22.524050 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:22.524542 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:22.524547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:22.524551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:22.525070 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:22.525385 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:22.525663 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 19:47:22.526021 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 19:47:22.527240 8783 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:22.619580 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 8659
I20260806 19:47:22.639415 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 8794
Aug 06 19:47:22 dist-test-slave-77v9 krb5kdc[8416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:47:22.659613 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:22.640241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.21.202.67:35419)
0806 19:47:22.640436 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:22.640443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:22.640687 (+ 244us) server_negotiation.cc:407] Connection header received
0806 19:47:22.640931 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:22.640935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:22.641018 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:22.641106 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:22.642933 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.643821 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.645864 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:22.646321 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:22.646429 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:22.652633 (+ 6204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:22.652648 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:22.652668 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:22.652709 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:22.656418 (+ 3709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:22.657281 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:22.657290 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:22.657292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:22.657458 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:22.657874 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:22.657878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:22.657880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:22.658291 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:22.658633 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:22.658950 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 19:47:22.659189 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 19:47:22.660290 8918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:22.789143 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 8794
I20260806 19:47:22.813872 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 8429
I20260806 19:47:22.815155 8491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:22.928778 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 8429
2026-08-06T19:47:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5025 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T19:47:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:22Z Disabled control of system clock
I20260806 19:47:23.023375 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:41757
--webserver_interface=127.21.202.126
--webserver_port=0
--builtin_ntp_servers=127.21.202.84:45371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:41757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 19:47:23.394707 8975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:23.395133 8975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:23.395337 8975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:23.403733 8975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:23.403878 8975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:23.403918 8975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:23.403975 8975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:23.404031 8975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:23.413628 8975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:41757
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:41757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:23.414987 8975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:23.416929 8975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:23.431474 8981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:23.434700 8980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:23.435451 8983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:23.438150 8975 server_base.cc:1034] running on GCE node
I20260806 19:47:23.439564 8975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:23.441599 8975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:23.442881 8975 hybrid_clock.cc:648] HybridClock initialized: now 1786045643442810 us; error 46 us; skew 500 ppm
I20260806 19:47:23.443326 8975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:47:23.446110 8975 webserver.cc:460] Webserver started at http://127.21.202.126:43709/ using document root <none> and password file <none>
I20260806 19:47:23.446877 8975 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:23.447016 8975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:23.447348 8975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:23.450860 8975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "72909014069c44d89c7f7f89b6811d0c"
format_stamp: "Formatted at 2026-08-06 19:47:23 on dist-test-slave-77v9"
server_key: "604614ca052a0f2f3ea0ce1d037f2f0d"
server_key_iv: "f8e683106631279322216f34d7801fb4"
server_key_version: "encryptionkey@0"
I20260806 19:47:23.451790 8975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "72909014069c44d89c7f7f89b6811d0c"
format_stamp: "Formatted at 2026-08-06 19:47:23 on dist-test-slave-77v9"
server_key: "604614ca052a0f2f3ea0ce1d037f2f0d"
server_key_iv: "f8e683106631279322216f34d7801fb4"
server_key_version: "encryptionkey@0"
I20260806 19:47:23.457506 8975 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 19:47:23.461318 8989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:23.462779 8975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:47:23.463016 8975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "72909014069c44d89c7f7f89b6811d0c"
format_stamp: "Formatted at 2026-08-06 19:47:23 on dist-test-slave-77v9"
server_key: "604614ca052a0f2f3ea0ce1d037f2f0d"
server_key_iv: "f8e683106631279322216f34d7801fb4"
server_key_version: "encryptionkey@0"
I20260806 19:47:23.463249 8975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:23.494369 8975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:23.495312 8975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:23.495631 8975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:23.518023 8975 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:41757
I20260806 19:47:23.518069 9041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:41757 every 8 connection(s)
I20260806 19:47:23.519611 8975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:23.523659 9042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:23.528647 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 8975
I20260806 19:47:23.528893 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:23.529364 22313 external_mini_cluster.cc:1428] Setting key 4a6c3ee02f002505148ae43729550527
I20260806 19:47:23.535367 9042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c: Bootstrap starting.
I20260806 19:47:23.539425 9042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:23.540673 9042 log.cc:824] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:23.544750 9042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c: No bootstrap required, opened a new log
I20260806 19:47:23.546217 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:23.531492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41084 (local address 127.21.202.126:41757)
0806 19:47:23.532593 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:23.532603 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:23.532652 (+ 49us) server_negotiation.cc:407] Connection header received
0806 19:47:23.534537 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:23.534562 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:23.535031 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:23.535370 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:23.536073 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:23.537520 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:23.539609 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:23.540363 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:23.540501 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:23.542030 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:23.542045 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:23.542074 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:23.542918 (+ 844us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:23.543012 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:23.543039 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:23.543134 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:23.543386 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 19:47:23.544501 (+ 1115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":448,"threads_started":1}
I20260806 19:47:23.553076 9042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72909014069c44d89c7f7f89b6811d0c" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41757 } }
I20260806 19:47:23.553570 9042 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:23.553692 9042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72909014069c44d89c7f7f89b6811d0c, State: Initialized, Role: FOLLOWER
I20260806 19:47:23.554407 9042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72909014069c44d89c7f7f89b6811d0c" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41757 } }
I20260806 19:47:23.554702 9042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:23.554872 9042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:23.555060 9042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:23.557391 9042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72909014069c44d89c7f7f89b6811d0c" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41757 } }
I20260806 19:47:23.557920 9042 leader_election.cc:302] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72909014069c44d89c7f7f89b6811d0c; no voters:
I20260806 19:47:23.558620 9042 leader_election.cc:288] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:23.558787 9048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:23.559594 9048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [term 1 LEADER]: Becoming Leader. State: Replica: 72909014069c44d89c7f7f89b6811d0c, State: Running, Role: LEADER
I20260806 19:47:23.560247 9048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72909014069c44d89c7f7f89b6811d0c" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41757 } }
I20260806 19:47:23.561098 9042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:23.563608 9049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72909014069c44d89c7f7f89b6811d0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72909014069c44d89c7f7f89b6811d0c" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41757 } } }
I20260806 19:47:23.564102 9049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:23.564358 9050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72909014069c44d89c7f7f89b6811d0c. Latest consensus state: current_term: 1 leader_uuid: "72909014069c44d89c7f7f89b6811d0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72909014069c44d89c7f7f89b6811d0c" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41757 } } }
I20260806 19:47:23.564755 9050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:23.565969 9057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:23.572117 9057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:23.582458 9057 catalog_manager.cc:1347] Generated new cluster ID: fa9f287665894661abeca52a32eacfef
I20260806 19:47:23.582592 9057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:23.603783 9057 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:23.605325 9057 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:23.615005 9057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c: Generated new TSK 0
I20260806 19:47:23.615988 9057 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:23.625782 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41757
--builtin_ntp_servers=127.21.202.84:45371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 19:47:23.991292 9068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:23.991591 9068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:23.991755 9068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:23.999228 9068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:23.999397 9068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:23.999594 9068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:47:24.009207 9068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:24.010632 9068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:24.013038 9068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:24.027338 9073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:24.027429 9076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:24.027384 9074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:24.028504 9068 server_base.cc:1034] running on GCE node
I20260806 19:47:24.029342 9068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:24.030496 9068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:24.031804 9068 hybrid_clock.cc:648] HybridClock initialized: now 1786045644031749 us; error 45 us; skew 500 ppm
I20260806 19:47:24.032198 9068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:47:24.035604 9068 webserver.cc:460] Webserver started at http://127.21.202.65:35899/ using document root <none> and password file <none>
I20260806 19:47:24.036541 9068 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:24.036710 9068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:24.037146 9068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:24.040164 9068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "b42ae54b3dfc4f788889e514ccf91069"
format_stamp: "Formatted at 2026-08-06 19:47:24 on dist-test-slave-77v9"
server_key: "941bc6e0cc2e1d5c7d42a342ab3f901d"
server_key_iv: "54a355a65881ba64748123f7e1026f4d"
server_key_version: "encryptionkey@0"
I20260806 19:47:24.041123 9068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "b42ae54b3dfc4f788889e514ccf91069"
format_stamp: "Formatted at 2026-08-06 19:47:24 on dist-test-slave-77v9"
server_key: "941bc6e0cc2e1d5c7d42a342ab3f901d"
server_key_iv: "54a355a65881ba64748123f7e1026f4d"
server_key_version: "encryptionkey@0"
I20260806 19:47:24.046993 9068 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:47:24.050946 9082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:24.052552 9068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:47:24.052750 9068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "b42ae54b3dfc4f788889e514ccf91069"
format_stamp: "Formatted at 2026-08-06 19:47:24 on dist-test-slave-77v9"
server_key: "941bc6e0cc2e1d5c7d42a342ab3f901d"
server_key_iv: "54a355a65881ba64748123f7e1026f4d"
server_key_version: "encryptionkey@0"
I20260806 19:47:24.053062 9068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:24.084793 9068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:24.085820 9068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:24.086145 9068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:24.087257 9068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:24.089205 9068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:24.089402 9068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:24.089594 9068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:24.089710 9068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:24.126313 9068 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:37113
I20260806 19:47:24.126403 9195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:37113 every 8 connection(s)
I20260806 19:47:24.127621 9068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:47:24.133260 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 9068
I20260806 19:47:24.133453 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:47:24.134042 22313 external_mini_cluster.cc:1428] Setting key be31eccae6043776576889688115ba37
I20260806 19:47:24.137260 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41757
--builtin_ntp_servers=127.21.202.84:45371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 19:47:24.142616 9198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:24.130355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:53459 (local address 127.21.202.126:41757)
0806 19:47:24.130886 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:24.130890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:24.132010 (+ 1120us) server_negotiation.cc:407] Connection header received
0806 19:47:24.134502 (+ 2492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:24.134507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:24.134581 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:24.134729 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:24.135974 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:24.136772 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:24.139454 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:24.139879 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:24.139995 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:24.141756 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:24.141762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:24.141765 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:24.141786 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:24.141832 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:24.141844 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:24.141929 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:24.142244 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 19:47:24.142345 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":250,"threads_started":1}
I20260806 19:47:24.144153 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:24.130950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:41757 (local address 127.21.202.65:53459)
0806 19:47:24.131836 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:24.131878 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:24.134179 (+ 2301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:24.134979 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:24.134993 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:24.135114 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:24.135743 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:24.135795 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:24.137005 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:24.137011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:24.139274 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:24.139295 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:24.140129 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:24.140139 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:24.140490 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:24.141334 (+ 844us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:47:24.141339 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:24.141413 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:47:24.141525 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:24.141957 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:24.141971 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:24.141982 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 19:47:24.142190 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 19:47:24.142652 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":277,"threads_started":1}
I20260806 19:47:24.145895 9196 heartbeater.cc:347] Connected to a master server at 127.21.202.126:41757
I20260806 19:47:24.146221 9196 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:24.147032 9196 heartbeater.cc:511] Master 127.21.202.126:41757 requested a full tablet report, sending...
I20260806 19:47:24.149175 9006 ts_manager.cc:194] Registered new tserver with Master: b42ae54b3dfc4f788889e514ccf91069 (127.21.202.65:37113)
I20260806 19:47:24.150686 9006 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.202.65:53459
W20260806 19:47:24.499194 9200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:24.499528 9200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:24.499665 9200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:24.507473 9200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:24.507606 9200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:24.507790 9200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:47:24.517675 9200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:24.518983 9200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:24.520948 9200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:24.535090 9205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:24.535183 9209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:24.535204 9207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:24.538856 9200 server_base.cc:1034] running on GCE node
I20260806 19:47:24.539582 9200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:24.540599 9200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:24.541846 9200 hybrid_clock.cc:648] HybridClock initialized: now 1786045644541761 us; error 60 us; skew 500 ppm
I20260806 19:47:24.542212 9200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:47:24.545017 9200 webserver.cc:460] Webserver started at http://127.21.202.66:36549/ using document root <none> and password file <none>
I20260806 19:47:24.545815 9200 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:24.545922 9200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:24.546314 9200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:24.549857 9200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "f504a92fe93846fc97a54ea437666eab"
format_stamp: "Formatted at 2026-08-06 19:47:24 on dist-test-slave-77v9"
server_key: "f1af68126112455f1ae635095383f7b4"
server_key_iv: "9cbb0a1cb9f71fd2ec0e04e0847be5b7"
server_key_version: "encryptionkey@0"
I20260806 19:47:24.550941 9200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "f504a92fe93846fc97a54ea437666eab"
format_stamp: "Formatted at 2026-08-06 19:47:24 on dist-test-slave-77v9"
server_key: "f1af68126112455f1ae635095383f7b4"
server_key_iv: "9cbb0a1cb9f71fd2ec0e04e0847be5b7"
server_key_version: "encryptionkey@0"
I20260806 19:47:24.557374 9200 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 19:47:24.561242 9215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:24.562801 9200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:47:24.562980 9200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "f504a92fe93846fc97a54ea437666eab"
format_stamp: "Formatted at 2026-08-06 19:47:24 on dist-test-slave-77v9"
server_key: "f1af68126112455f1ae635095383f7b4"
server_key_iv: "9cbb0a1cb9f71fd2ec0e04e0847be5b7"
server_key_version: "encryptionkey@0"
I20260806 19:47:24.563233 9200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:24.595816 9200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:24.596731 9200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:24.597229 9200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:24.598433 9200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:24.600103 9200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:24.600198 9200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:24.600546 9200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:24.600651 9200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:24.641243 9200 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:36975
I20260806 19:47:24.641366 9327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:36975 every 8 connection(s)
I20260806 19:47:24.642627 9200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:47:24.643275 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 9200
I20260806 19:47:24.643496 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:47:24.644153 22313 external_mini_cluster.cc:1428] Setting key db8542384b386f7530cc1f2379a9dd9e
I20260806 19:47:24.648102 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41757
--builtin_ntp_servers=127.21.202.84:45371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 19:47:24.657207 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:24.645749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:49119 (local address 127.21.202.126:41757)
0806 19:47:24.646376 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:24.646382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:24.647754 (+ 1372us) server_negotiation.cc:407] Connection header received
0806 19:47:24.649766 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:24.649772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:24.649870 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:24.649989 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:24.651117 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:24.651864 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:24.653553 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:24.653984 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:24.654138 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:24.656042 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:24.656048 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:24.656052 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:24.656074 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:24.656120 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:24.656134 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:24.656247 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:24.656636 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 19:47:24.656730 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":390,"threads_started":1}
I20260806 19:47:24.658811 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:24.646475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:41757 (local address 127.21.202.66:49119)
0806 19:47:24.647565 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:24.647611 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:24.649477 (+ 1866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:24.650240 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:24.650249 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:24.650342 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:24.650942 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:24.650959 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:24.652040 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:24.652045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:24.653398 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:24.653415 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:24.654235 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:24.654243 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:24.654483 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:24.655423 (+ 940us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:47:24.655428 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:24.655494 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:47:24.655626 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:24.656281 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:24.656298 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:24.656311 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 19:47:24.656586 (+ 275us) client_negotiation.cc:240] Negotiation successful
0806 19:47:24.657241 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":382,"threads_started":1}
I20260806 19:47:24.660538 9328 heartbeater.cc:347] Connected to a master server at 127.21.202.126:41757
I20260806 19:47:24.660975 9328 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:24.662060 9328 heartbeater.cc:511] Master 127.21.202.126:41757 requested a full tablet report, sending...
I20260806 19:47:24.663664 9006 ts_manager.cc:194] Registered new tserver with Master: f504a92fe93846fc97a54ea437666eab (127.21.202.66:36975)
I20260806 19:47:24.664664 9006 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.202.66:49119
W20260806 19:47:25.037886 9332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:25.038213 9332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:25.038368 9332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:25.046550 9332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:25.046682 9332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:25.047048 9332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:47:25.058403 9332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:45371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:41757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:25.060010 9332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:25.062005 9332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:25.077730 9337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:25.077821 9338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:25.077987 9340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:25.078984 9332 server_base.cc:1034] running on GCE node
I20260806 19:47:25.079707 9332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:25.080852 9332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:25.082151 9332 hybrid_clock.cc:648] HybridClock initialized: now 1786045645082084 us; error 55 us; skew 500 ppm
I20260806 19:47:25.082572 9332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:47:25.089697 9332 webserver.cc:460] Webserver started at http://127.21.202.67:37487/ using document root <none> and password file <none>
I20260806 19:47:25.090561 9332 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:25.090708 9332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:25.091063 9332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:25.094252 9332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "392865b892454f158b4a86128e61526e"
format_stamp: "Formatted at 2026-08-06 19:47:25 on dist-test-slave-77v9"
server_key: "b2300f40b1a39eaa34bafe5d2005303f"
server_key_iv: "dd1847c270268d56a979147e203777a7"
server_key_version: "encryptionkey@0"
I20260806 19:47:25.095198 9332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "392865b892454f158b4a86128e61526e"
format_stamp: "Formatted at 2026-08-06 19:47:25 on dist-test-slave-77v9"
server_key: "b2300f40b1a39eaa34bafe5d2005303f"
server_key_iv: "dd1847c270268d56a979147e203777a7"
server_key_version: "encryptionkey@0"
I20260806 19:47:25.100752 9332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 19:47:25.105069 9347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:25.106616 9332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:47:25.106859 9332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "392865b892454f158b4a86128e61526e"
format_stamp: "Formatted at 2026-08-06 19:47:25 on dist-test-slave-77v9"
server_key: "b2300f40b1a39eaa34bafe5d2005303f"
server_key_iv: "dd1847c270268d56a979147e203777a7"
server_key_version: "encryptionkey@0"
I20260806 19:47:25.107115 9332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:25.154201 9196 heartbeater.cc:503] Master 127.21.202.126:41757 was elected leader, sending a full tablet report...
I20260806 19:47:25.165679 9332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:25.166687 9332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:25.167032 9332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:25.168258 9332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:25.169872 9332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:25.169987 9332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:25.170136 9332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:25.170203 9332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:25.207432 9332 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:36553
I20260806 19:47:25.207464 9459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:36553 every 8 connection(s)
I20260806 19:47:25.208828 9332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:47:25.217883 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 9332
I20260806 19:47:25.218065 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:47:25.218566 22313 external_mini_cluster.cc:1428] Setting key 981a256a9b89b4801e90d4770a2f1a15
I20260806 19:47:25.222443 9462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.211671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:52687 (local address 127.21.202.126:41757)
0806 19:47:25.212215 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:25.212219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:25.213396 (+ 1177us) server_negotiation.cc:407] Connection header received
0806 19:47:25.215234 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:25.215240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:25.215339 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:25.215460 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:25.216530 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.217366 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.219137 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.219551 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.219658 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.221354 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:25.221360 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:25.221363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:25.221383 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:25.221425 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:25.221470 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:25.221590 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:25.221929 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 19:47:25.222031 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":306,"threads_started":1}
I20260806 19:47:25.223418 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.212160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:41757 (local address 127.21.202.67:52687)
0806 19:47:25.213233 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.213276 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:25.214963 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:25.215697 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:25.215705 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:25.215796 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:25.216351 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.216371 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.217544 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.217548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.218959 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.218971 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.219798 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.219804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.220095 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.220949 (+ 854us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:47:25.220953 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:25.221013 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:47:25.221118 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:25.221634 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:25.221646 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:25.221654 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 19:47:25.221918 (+ 264us) client_negotiation.cc:240] Negotiation successful
0806 19:47:25.222322 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":320,"threads_started":1}
I20260806 19:47:25.225229 9460 heartbeater.cc:347] Connected to a master server at 127.21.202.126:41757
I20260806 19:47:25.225561 9460 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:25.226379 9460 heartbeater.cc:511] Master 127.21.202.126:41757 requested a full tablet report, sending...
I20260806 19:47:25.228184 9006 ts_manager.cc:194] Registered new tserver with Master: 392865b892454f158b4a86128e61526e (127.21.202.67:36553)
I20260806 19:47:25.229108 9006 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.202.67:52687
I20260806 19:47:25.234730 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:47:25.246937 9462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.239697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41096 (local address 127.21.202.126:41757)
0806 19:47:25.240091 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:25.240103 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:25.240553 (+ 450us) server_negotiation.cc:407] Connection header received
0806 19:47:25.240743 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:25.240748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:25.240897 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:25.240995 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:25.241577 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.242332 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.243775 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.244351 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.244467 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.246069 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:25.246074 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:25.246078 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:25.246098 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:25.246133 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:25.246146 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:25.246266 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:25.246540 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 19:47:25.246610 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260806 19:47:25.251518 9006 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:47:25.258844 9006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:47:25.288586 9473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.273526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55022)
0806 19:47:25.275057 (+ 1531us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.275111 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:25.275316 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:25.276655 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:25.276667 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:25.276712 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:25.277420 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.277437 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.280154 (+ 2717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.280164 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.283805 (+ 3641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.283831 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.287164 (+ 3333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.287174 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.287518 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.287596 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 19:47:25.287780 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 19:47:25.288083 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1225,"thread_start_us":1061,"threads_started":1}
I20260806 19:47:25.290059 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.273703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.21.202.65:37113)
0806 19:47:25.274578 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:25.274590 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:25.275558 (+ 968us) server_negotiation.cc:407] Connection header received
0806 19:47:25.275690 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:25.275702 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:25.276049 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:25.276272 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:25.277710 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.279899 (+ 2189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.284163 (+ 4264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.286355 (+ 2192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.286562 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.286990 (+ 428us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:25.289339 (+ 2349us) server_negotiation.cc:299] Negotiation successful
0806 19:47:25.289621 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":539,"threads_started":1}
I20260806 19:47:25.299113 9473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.275554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:36975 (local address 127.0.0.1:43172)
0806 19:47:25.288975 (+ 13421us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.288999 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:25.289126 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:25.291537 (+ 2411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:25.291541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:25.291569 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:25.291848 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.291861 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.293778 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.293788 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.295901 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.295917 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.298172 (+ 2255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.298179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.298436 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.298481 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 19:47:25.298577 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 19:47:25.298745 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13231,"mutex_wait_us":244,"thread_start_us":18307,"threads_started":1}
I20260806 19:47:25.298702 9131 tablet_service.cc:1467] Processing CreateTablet for tablet 98ba4c8fb53f47af8c4f93b67eecb1ac (DEFAULT_TABLE table=test-table [id=f1f62f676a024bc18b377bdfbc19f512]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:25.299353 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.275071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36290 (local address 127.21.202.67:36553)
0806 19:47:25.290281 (+ 15210us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:25.290291 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:25.290331 (+ 40us) server_negotiation.cc:407] Connection header received
0806 19:47:25.290439 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:25.290447 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:25.290739 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:25.290962 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:25.292175 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.293749 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.296422 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.297831 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.297932 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.298169 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:25.298761 (+ 592us) server_negotiation.cc:299] Negotiation successful
0806 19:47:25.298948 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14993,"thread_start_us":15233,"threads_started":1}
I20260806 19:47:25.299167 9478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.276158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43172 (local address 127.21.202.66:36975)
0806 19:47:25.290952 (+ 14794us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:25.290960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:25.290989 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:47:25.291101 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:25.291109 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:25.291309 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:25.291753 (+ 444us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:25.291993 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.293579 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.296087 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.297655 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.297814 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.298064 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:25.298665 (+ 601us) server_negotiation.cc:299] Negotiation successful
0806 19:47:25.298845 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14616,"thread_start_us":14490,"threads_started":1}
I20260806 19:47:25.300016 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.274335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:36553 (local address 127.0.0.1:36290)
0806 19:47:25.288169 (+ 13834us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.288195 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:25.288348 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:25.291261 (+ 2913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:25.291266 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:25.291294 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:25.291975 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.291985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.293998 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.294003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.296166 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.296182 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.298363 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.298369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.298576 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.298606 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 19:47:25.299539 (+ 933us) client_negotiation.cc:240] Negotiation successful
0806 19:47:25.299723 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13643,"mutex_wait_us":26,"thread_start_us":14104,"threads_started":1}
I20260806 19:47:25.300321 9131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ba4c8fb53f47af8c4f93b67eecb1ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:25.303155 9395 tablet_service.cc:1467] Processing CreateTablet for tablet 98ba4c8fb53f47af8c4f93b67eecb1ac (DEFAULT_TABLE table=test-table [id=f1f62f676a024bc18b377bdfbc19f512]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:25.303716 9263 tablet_service.cc:1467] Processing CreateTablet for tablet 98ba4c8fb53f47af8c4f93b67eecb1ac (DEFAULT_TABLE table=test-table [id=f1f62f676a024bc18b377bdfbc19f512]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:25.304270 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ba4c8fb53f47af8c4f93b67eecb1ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:25.305171 9263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ba4c8fb53f47af8c4f93b67eecb1ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:25.315977 9480 tablet_bootstrap.cc:492] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e: Bootstrap starting.
I20260806 19:47:25.319195 9480 tablet_bootstrap.cc:654] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:25.320271 9480 log.cc:824] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:25.321331 9479 tablet_bootstrap.cc:492] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069: Bootstrap starting.
I20260806 19:47:25.323200 9482 tablet_bootstrap.cc:492] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab: Bootstrap starting.
I20260806 19:47:25.324939 9480 tablet_bootstrap.cc:492] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e: No bootstrap required, opened a new log
I20260806 19:47:25.325223 9480 ts_tablet_manager.cc:1397] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.007s
I20260806 19:47:25.325745 9479 tablet_bootstrap.cc:654] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:25.326749 9482 tablet_bootstrap.cc:654] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:25.327139 9479 log.cc:824] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:25.328029 9482 log.cc:824] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:25.330768 9479 tablet_bootstrap.cc:492] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069: No bootstrap required, opened a new log
I20260806 19:47:25.331058 9479 ts_tablet_manager.cc:1397] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.002s
I20260806 19:47:25.333098 9482 tablet_bootstrap.cc:492] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab: No bootstrap required, opened a new log
I20260806 19:47:25.333374 9482 ts_tablet_manager.cc:1397] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260806 19:47:25.333993 9480 raft_consensus.cc:348] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b42ae54b3dfc4f788889e514ccf91069" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37113 } } peers { permanent_uuid: "f504a92fe93846fc97a54ea437666eab" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 36975 } } peers { permanent_uuid: "392865b892454f158b4a86128e61526e" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36553 } }
I20260806 19:47:25.334372 9480 raft_consensus.cc:374] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:25.334515 9480 raft_consensus.cc:729] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 392865b892454f158b4a86128e61526e, State: Initialized, Role: FOLLOWER
I20260806 19:47:25.335152 9480 consensus_queue.cc:260] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b42ae54b3dfc4f788889e514ccf91069" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37113 } } peers { permanent_uuid: "f504a92fe93846fc97a54ea437666eab" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 36975 } } peers { permanent_uuid: "392865b892454f158b4a86128e61526e" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36553 } }
I20260806 19:47:25.336297 9460 heartbeater.cc:503] Master 127.21.202.126:41757 was elected leader, sending a full tablet report...
I20260806 19:47:25.336971 9480 ts_tablet_manager.cc:1428] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e: Time spent starting tablet: real 0.012s user 0.003s sys 0.009s
I20260806 19:47:25.343567 9479 raft_consensus.cc:348] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b42ae54b3dfc4f788889e514ccf91069" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37113 } } peers { permanent_uuid: "f504a92fe93846fc97a54ea437666eab" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 36975 } } peers { permanent_uuid: "392865b892454f158b4a86128e61526e" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36553 } }
I20260806 19:47:25.344067 9479 raft_consensus.cc:374] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:25.344184 9479 raft_consensus.cc:729] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b42ae54b3dfc4f788889e514ccf91069, State: Initialized, Role: FOLLOWER
I20260806 19:47:25.344318 9482 raft_consensus.cc:348] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b42ae54b3dfc4f788889e514ccf91069" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37113 } } peers { permanent_uuid: "f504a92fe93846fc97a54ea437666eab" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 36975 } } peers { permanent_uuid: "392865b892454f158b4a86128e61526e" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36553 } }
I20260806 19:47:25.344729 9482 raft_consensus.cc:374] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:25.344866 9482 raft_consensus.cc:729] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f504a92fe93846fc97a54ea437666eab, State: Initialized, Role: FOLLOWER
I20260806 19:47:25.344892 9479 consensus_queue.cc:260] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b42ae54b3dfc4f788889e514ccf91069" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37113 } } peers { permanent_uuid: "f504a92fe93846fc97a54ea437666eab" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 36975 } } peers { permanent_uuid: "392865b892454f158b4a86128e61526e" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36553 } }
I20260806 19:47:25.345464 9482 consensus_queue.cc:260] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b42ae54b3dfc4f788889e514ccf91069" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37113 } } peers { permanent_uuid: "f504a92fe93846fc97a54ea437666eab" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 36975 } } peers { permanent_uuid: "392865b892454f158b4a86128e61526e" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36553 } }
I20260806 19:47:25.346580 9328 heartbeater.cc:503] Master 127.21.202.126:41757 was elected leader, sending a full tablet report...
I20260806 19:47:25.346915 9479 ts_tablet_manager.cc:1428] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069: Time spent starting tablet: real 0.016s user 0.008s sys 0.005s
I20260806 19:47:25.348428 9482 ts_tablet_manager.cc:1428] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
W20260806 19:47:25.380412 9197 tablet.cc:2369] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:47:25.395231 9329 tablet.cc:2369] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:47:25.418543 9487 raft_consensus.cc:482] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:47:25.418903 9487 raft_consensus.cc:504] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b42ae54b3dfc4f788889e514ccf91069" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37113 } } peers { permanent_uuid: "f504a92fe93846fc97a54ea437666eab" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 36975 } } peers { permanent_uuid: "392865b892454f158b4a86128e61526e" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36553 } }
I20260806 19:47:25.420753 9487 leader_election.cc:288] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f504a92fe93846fc97a54ea437666eab (127.21.202.66:36975), 392865b892454f158b4a86128e61526e (127.21.202.67:36553)
I20260806 19:47:25.423820 9486 raft_consensus.cc:482] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:47:25.424153 9486 raft_consensus.cc:504] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b42ae54b3dfc4f788889e514ccf91069" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37113 } } peers { permanent_uuid: "f504a92fe93846fc97a54ea437666eab" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 36975 } } peers { permanent_uuid: "392865b892454f158b4a86128e61526e" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36553 } }
I20260806 19:47:25.426019 9486 leader_election.cc:288] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b42ae54b3dfc4f788889e514ccf91069 (127.21.202.65:37113), f504a92fe93846fc97a54ea437666eab (127.21.202.66:36975)
I20260806 19:47:25.430768 9478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.421277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:55717 (local address 127.21.202.66:36975)
0806 19:47:25.421616 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:25.421624 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:25.422479 (+ 855us) server_negotiation.cc:407] Connection header received
0806 19:47:25.422641 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:25.422649 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:25.422753 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:25.422891 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:25.423525 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.424554 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.427657 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.428897 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.429053 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.429143 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:25.430200 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0806 19:47:25.430403 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260806 19:47:25.431728 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.421238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:36975 (local address 127.21.202.65:55717)
0806 19:47:25.422367 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.422383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:25.422516 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:25.422904 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:25.422909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:25.422932 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:25.423370 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.423380 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.424712 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.424717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.427401 (+ 2684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.427416 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.429323 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.429331 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.429945 (+ 614us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.430003 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 19:47:25.431041 (+ 1038us) client_negotiation.cc:240] Negotiation successful
0806 19:47:25.431239 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1007,"mutex_wait_us":127,"thread_start_us":530,"threads_started":1}
I20260806 19:47:25.433045 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ba4c8fb53f47af8c4f93b67eecb1ac" candidate_uuid: "b42ae54b3dfc4f788889e514ccf91069" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f504a92fe93846fc97a54ea437666eab" is_pre_election: true
I20260806 19:47:25.433780 9283 raft_consensus.cc:2454] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b42ae54b3dfc4f788889e514ccf91069 in term 0.
I20260806 19:47:25.434772 9085 leader_election.cc:302] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b42ae54b3dfc4f788889e514ccf91069, f504a92fe93846fc97a54ea437666eab; no voters:
I20260806 19:47:25.435359 9487 raft_consensus.cc:2781] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:47:25.435535 9487 raft_consensus.cc:482] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:47:25.435668 9487 raft_consensus.cc:3037] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:25.436578 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.421372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:36553 (local address 127.21.202.65:60387)
0806 19:47:25.422585 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.422603 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:25.422738 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:25.423100 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:25.423103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:25.423125 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:25.423562 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.423576 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.425065 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.425070 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.427410 (+ 2340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.427418 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.435620 (+ 8202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.435626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.435833 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.435870 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:47:25.435966 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 19:47:25.436123 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1081,"mutex_wait_us":47,"thread_start_us":555,"threads_started":1}
I20260806 19:47:25.438745 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.426440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:46007 (local address 127.21.202.65:37113)
0806 19:47:25.426682 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:25.426690 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:25.427344 (+ 654us) server_negotiation.cc:407] Connection header received
0806 19:47:25.427726 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:25.427732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:25.427909 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:25.428093 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:25.430872 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.433547 (+ 2675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.436050 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.437419 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.437609 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.437676 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:25.438389 (+ 713us) server_negotiation.cc:299] Negotiation successful
0806 19:47:25.438524 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 19:47:25.438822 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.426377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.21.202.67:46007)
0806 19:47:25.427226 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.427246 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:25.427505 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:25.428152 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:25.428158 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:25.428188 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:25.430609 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.430628 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.433793 (+ 3165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.433834 (+ 41us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.435840 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.435856 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.437919 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.437926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.438170 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.438219 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 19:47:25.438348 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 19:47:25.438520 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":1948}
I20260806 19:47:25.439404 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.421330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:60387 (local address 127.21.202.67:36553)
0806 19:47:25.421542 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:25.421550 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:25.422712 (+ 1162us) server_negotiation.cc:407] Connection header received
0806 19:47:25.422882 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:25.422887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:25.422965 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:25.423069 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:25.423813 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.424927 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.428400 (+ 3473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.430035 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.430210 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.430293 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:25.439025 (+ 8732us) server_negotiation.cc:299] Negotiation successful
0806 19:47:25.439168 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":54}
I20260806 19:47:25.439263 9487 raft_consensus.cc:504] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b42ae54b3dfc4f788889e514ccf91069" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37113 } } peers { permanent_uuid: "f504a92fe93846fc97a54ea437666eab" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 36975 } } peers { permanent_uuid: "392865b892454f158b4a86128e61526e" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36553 } }
I20260806 19:47:25.440249 9487 leader_election.cc:288] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [CANDIDATE]: Term 1 election: Requested vote from peers f504a92fe93846fc97a54ea437666eab (127.21.202.66:36975), 392865b892454f158b4a86128e61526e (127.21.202.67:36553)
I20260806 19:47:25.440336 9151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ba4c8fb53f47af8c4f93b67eecb1ac" candidate_uuid: "392865b892454f158b4a86128e61526e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b42ae54b3dfc4f788889e514ccf91069" is_pre_election: true
I20260806 19:47:25.440716 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ba4c8fb53f47af8c4f93b67eecb1ac" candidate_uuid: "b42ae54b3dfc4f788889e514ccf91069" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f504a92fe93846fc97a54ea437666eab"
I20260806 19:47:25.440971 9283 raft_consensus.cc:3037] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:25.441308 9414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ba4c8fb53f47af8c4f93b67eecb1ac" candidate_uuid: "b42ae54b3dfc4f788889e514ccf91069" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "392865b892454f158b4a86128e61526e"
I20260806 19:47:25.441391 9415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ba4c8fb53f47af8c4f93b67eecb1ac" candidate_uuid: "b42ae54b3dfc4f788889e514ccf91069" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "392865b892454f158b4a86128e61526e" is_pre_election: true
I20260806 19:47:25.441744 9414 raft_consensus.cc:3037] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:25.442437 9151 raft_consensus.cc:2379] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 392865b892454f158b4a86128e61526e in current term 1: Already voted for candidate b42ae54b3dfc4f788889e514ccf91069 in this term.
I20260806 19:47:25.444754 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.428965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:36975 (local address 127.21.202.67:38929)
0806 19:47:25.437254 (+ 8289us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.437274 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:25.437485 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:25.437851 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:25.437859 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:25.437893 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:25.438315 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.438329 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.439753 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.439757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.441917 (+ 2160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:25.441933 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.443959 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.443967 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:25.444150 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.444185 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 19:47:25.444299 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 19:47:25.444448 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8145,"thread_start_us":7885,"threads_started":1}
I20260806 19:47:25.444837 9478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.431828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:38929 (local address 127.21.202.66:36975)
0806 19:47:25.435471 (+ 3643us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:25.435477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:25.437383 (+ 1906us) server_negotiation.cc:407] Connection header received
0806 19:47:25.437601 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:25.437606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:25.437696 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:25.437794 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:25.438511 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.439570 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.442174 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.443561 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.443702 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.443777 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:25.444357 (+ 580us) server_negotiation.cc:299] Negotiation successful
0806 19:47:25.444496 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3517}
I20260806 19:47:25.444980 9283 raft_consensus.cc:2454] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b42ae54b3dfc4f788889e514ccf91069 in term 1.
I20260806 19:47:25.445344 9414 raft_consensus.cc:2454] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b42ae54b3dfc4f788889e514ccf91069 in term 1.
I20260806 19:47:25.445433 9085 leader_election.cc:302] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b42ae54b3dfc4f788889e514ccf91069, f504a92fe93846fc97a54ea437666eab; no voters:
I20260806 19:47:25.445709 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ba4c8fb53f47af8c4f93b67eecb1ac" candidate_uuid: "392865b892454f158b4a86128e61526e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f504a92fe93846fc97a54ea437666eab" is_pre_election: true
I20260806 19:47:25.446136 9283 raft_consensus.cc:2379] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 392865b892454f158b4a86128e61526e in current term 1: Already voted for candidate b42ae54b3dfc4f788889e514ccf91069 in this term.
I20260806 19:47:25.446154 9487 raft_consensus.cc:2781] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:25.446825 9487 raft_consensus.cc:686] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [term 1 LEADER]: Becoming Leader. State: Replica: b42ae54b3dfc4f788889e514ccf91069, State: Running, Role: LEADER
I20260806 19:47:25.447037 9350 leader_election.cc:302] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 392865b892454f158b4a86128e61526e; no voters: b42ae54b3dfc4f788889e514ccf91069, f504a92fe93846fc97a54ea437666eab
I20260806 19:47:25.447492 9487 consensus_queue.cc:237] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b42ae54b3dfc4f788889e514ccf91069" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37113 } } peers { permanent_uuid: "f504a92fe93846fc97a54ea437666eab" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 36975 } } peers { permanent_uuid: "392865b892454f158b4a86128e61526e" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36553 } }
I20260806 19:47:25.447557 9486 raft_consensus.cc:2726] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 19:47:25.452958 9005 catalog_manager.cc:5495] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 reported cstate change: term changed from 0 to 1, leader changed from <none> to b42ae54b3dfc4f788889e514ccf91069 (127.21.202.65). New cstate: current_term: 1 leader_uuid: "b42ae54b3dfc4f788889e514ccf91069" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b42ae54b3dfc4f788889e514ccf91069" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 37113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f504a92fe93846fc97a54ea437666eab" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 36975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "392865b892454f158b4a86128e61526e" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36553 } health_report { overall_health: UNKNOWN } } }
W20260806 19:47:25.460619 9461 tablet.cc:2369] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:47:25.491698 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.485100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55024 (local address 127.21.202.65:37113)
0806 19:47:25.485291 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:25.485296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:25.485317 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:47:25.485576 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:25.485584 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:25.485716 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:25.485903 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:47:25.486404 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.487258 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.489262 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.489695 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.489798 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.490381 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:47:25.490926 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:47:25.491062 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:25.491288 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 19:47:25.491329 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 19:47:25.501362 9414 raft_consensus.cc:1261] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [term 1 FOLLOWER]: Refusing update from remote peer b42ae54b3dfc4f788889e514ccf91069: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:47:25.501362 9283 raft_consensus.cc:1261] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab [term 1 FOLLOWER]: Refusing update from remote peer b42ae54b3dfc4f788889e514ccf91069: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:47:25.502291 9492 consensus_queue.cc:1035] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f504a92fe93846fc97a54ea437666eab" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 36975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:47:25.502704 9487 consensus_queue.cc:1035] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069 [LEADER]: Connected to new peer: Peer: permanent_uuid: "392865b892454f158b4a86128e61526e" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:47:25.532217 9005 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:47:25.532549 9005 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:47:25.537324 9005 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72909014069c44d89c7f7f89b6811d0c: Sending DeleteTablet for 3 replicas of tablet 98ba4c8fb53f47af8c4f93b67eecb1ac
I20260806 19:47:25.539078 9263 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ba4c8fb53f47af8c4f93b67eecb1ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:25 UTC) from {username='slave'} at 127.0.0.1:43172
I20260806 19:47:25.539467 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 9068
I20260806 19:47:25.540272 9395 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ba4c8fb53f47af8c4f93b67eecb1ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:25 UTC) from {username='slave'} at 127.0.0.1:36290
I20260806 19:47:25.542237 9130 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ba4c8fb53f47af8c4f93b67eecb1ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:25 UTC) from {username='slave'} at 127.0.0.1:55022
I20260806 19:47:25.542855 9131 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ba4c8fb53f47af8c4f93b67eecb1ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:25 UTC) from {username='slave'} at 127.0.0.1:55022
I20260806 19:47:25.562233 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.541189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.21.202.65:37113)
0806 19:47:25.541387 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:25.541393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:25.552370 (+ 10977us) server_negotiation.cc:407] Connection header received
0806 19:47:25.552707 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:25.552714 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:25.552885 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:25.552978 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:25.553720 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.555094 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.557316 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.558118 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.558226 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.559799 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:25.559810 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:25.559830 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:25.560844 (+ 1014us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:25.560956 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:25.560989 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:25.561084 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:25.561727 (+ 643us) server_negotiation.cc:299] Negotiation successful
0806 19:47:25.561844 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 19:47:25.563058 9191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:25.567478 9505 tablet_replica.cc:330] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e: stopping tablet replica
I20260806 19:47:25.568619 9504 tablet_replica.cc:330] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab: stopping tablet replica
I20260806 19:47:25.569159 9504 raft_consensus.cc:2229] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:47:25.569744 9504 raft_consensus.cc:2258] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:25.570235 9505 raft_consensus.cc:2229] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:47:25.570711 9505 raft_consensus.cc:2258] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:25.574465 9504 ts_tablet_manager.cc:1905] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:25.574553 9505 ts_tablet_manager.cc:1905] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:25.579738 9504 ts_tablet_manager.cc:1918] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:25.579946 9504 log.cc:1196] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/98ba4c8fb53f47af8c4f93b67eecb1ac
I20260806 19:47:25.580415 9504 ts_tablet_manager.cc:1939] T 98ba4c8fb53f47af8c4f93b67eecb1ac P f504a92fe93846fc97a54ea437666eab: Deleting consensus metadata
I20260806 19:47:25.581986 9505 ts_tablet_manager.cc:1918] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:25.582145 9505 log.cc:1196] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/98ba4c8fb53f47af8c4f93b67eecb1ac
I20260806 19:47:25.582222 8992 catalog_manager.cc:4841] TS f504a92fe93846fc97a54ea437666eab (127.21.202.66:36975): tablet 98ba4c8fb53f47af8c4f93b67eecb1ac (table test-table [id=f1f62f676a024bc18b377bdfbc19f512]) successfully deleted
I20260806 19:47:25.582691 9505 ts_tablet_manager.cc:1939] T 98ba4c8fb53f47af8c4f93b67eecb1ac P 392865b892454f158b4a86128e61526e: Deleting consensus metadata
I20260806 19:47:25.584679 8991 catalog_manager.cc:4841] TS 392865b892454f158b4a86128e61526e (127.21.202.67:36553): tablet 98ba4c8fb53f47af8c4f93b67eecb1ac (table test-table [id=f1f62f676a024bc18b377bdfbc19f512]) successfully deleted
I20260806 19:47:25.676497 9507 tablet_replica.cc:330] T 98ba4c8fb53f47af8c4f93b67eecb1ac P b42ae54b3dfc4f788889e514ccf91069: stopping tablet replica
I20260806 19:47:25.676517 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 9068
I20260806 19:47:25.696661 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 9200
W20260806 19:47:25.696781 8992 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.202.65:37113 (error 108)
I20260806 19:47:25.698905 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.698417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55036)
0806 19:47:25.698616 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.698715 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 19:47:25.699195 8992 catalog_manager.cc:4584] TS b42ae54b3dfc4f788889e514ccf91069 (127.21.202.65:37113): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 98ba4c8fb53f47af8c4f93b67eecb1ac: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
I20260806 19:47:25.704487 9478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.697630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43176 (local address 127.21.202.66:36975)
0806 19:47:25.697952 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:25.697958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:25.697985 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:47:25.698182 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:25.698189 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:25.698300 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:25.698406 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:25.698873 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.699768 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.701321 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.701749 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.701883 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.702984 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:25.703002 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:25.703016 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:25.703617 (+ 601us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:25.703669 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:25.703689 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:25.703780 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:25.704040 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 19:47:25.704170 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260806 19:47:25.705336 9323 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:25.744040 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.743484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55042)
0806 19:47:25.743730 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.743838 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260806 19:47:25.745740 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.745098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55054)
0806 19:47:25.745270 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.745383 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 19:47:25.753626 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.752976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55058)
0806 19:47:25.753173 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.753276 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 19:47:25.754758 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.754326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55066)
0806 19:47:25.754511 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.754625 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 19:47:25.794641 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.794032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55070)
0806 19:47:25.794297 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.794422 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260806 19:47:25.795840 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.795355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55082)
0806 19:47:25.795608 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.795705 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260806 19:47:25.801209 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 9200
I20260806 19:47:25.821561 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 9332
I20260806 19:47:25.829452 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.822752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.21.202.67:36553)
0806 19:47:25.823000 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:25.823008 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:25.823126 (+ 118us) server_negotiation.cc:407] Connection header received
0806 19:47:25.823349 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:25.823368 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:25.823462 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:25.823578 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:25.824015 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.824769 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.826157 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:25.826588 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:25.826731 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:25.827902 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:25.827911 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:25.827927 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:25.828546 (+ 619us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:25.828606 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:25.828627 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:25.828758 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:25.828994 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 19:47:25.829114 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 19:47:25.830137 9455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:25.831815 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.831194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55098)
0806 19:47:25.831498 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.831601 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
I20260806 19:47:25.832965 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.832515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55104)
0806 19:47:25.832693 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.832780 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 19:47:25.897116 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.896460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55112)
0806 19:47:25.896731 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.896887 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260806 19:47:25.898516 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.898113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55124)
0806 19:47:25.898277 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.898366 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 19:47:25.929526 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 9332
I20260806 19:47:25.935395 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.934702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55134)
0806 19:47:25.935020 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.935146 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 19:47:25.936693 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:25.936223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:37113 (local address 127.0.0.1:55142)
0806 19:47:25.936420 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:25.936523 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.65:37113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 19:47:25.950789 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 8975
I20260806 19:47:25.952149 9037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:26.067940 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 8975
2026-08-06T19:47:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3142 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:26 dist-test-slave-77v9 krb5kdc[9516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:26 dist-test-slave-77v9 krb5kdc[9516](info): set up 2 sockets
Aug 06 19:47:26 dist-test-slave-77v9 krb5kdc[9516](info): commencing operation
krb5kdc: starting...
W20260806 19:47:27.257392 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:27 dist-test-slave-77v9 krb5kdc[9516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:47:27.604528 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:40977
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:40977
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:27.966277 9529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:27.966620 9529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:27.966797 9529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:27.974257 9529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:27.974385 9529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:27.974470 9529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:27.974575 9529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:27.974653 9529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:27.984921 9529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:40977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:40977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:27.986351 9529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:27.987703 9529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:28.000654 9534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:28.000663 9535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:28.001309 9529 server_base.cc:1034] running on GCE node
W20260806 19:47:28.001533 9537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:28.002442 9529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:28.003942 9529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:28.005178 9529 hybrid_clock.cc:648] HybridClock initialized: now 1786045648005113 us; error 55 us; skew 500 ppm
I20260806 19:47:28.005553 9529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:28 dist-test-slave-77v9 krb5kdc[9516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045648, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:28.009063 9529 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:47:28.010982 9529 webserver.cc:460] Webserver started at http://127.21.202.126:40063/ using document root <none> and password file <none>
I20260806 19:47:28.011699 9529 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:28.011831 9529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:28.012193 9529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:28.015972 9529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "d37d06f486f64aa98e71be6b5947f13f"
format_stamp: "Formatted at 2026-08-06 19:47:28 on dist-test-slave-77v9"
server_key: "e1199a806d62f6836b97cfe34f7bcd02"
server_key_iv: "f8f71afbbb67b53607a9baa94fc784a9"
server_key_version: "encryptionkey@0"
I20260806 19:47:28.017066 9529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "d37d06f486f64aa98e71be6b5947f13f"
format_stamp: "Formatted at 2026-08-06 19:47:28 on dist-test-slave-77v9"
server_key: "e1199a806d62f6836b97cfe34f7bcd02"
server_key_iv: "f8f71afbbb67b53607a9baa94fc784a9"
server_key_version: "encryptionkey@0"
I20260806 19:47:28.022778 9529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 19:47:28.026839 9544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:28.028626 9529 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 19:47:28.028934 9529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "d37d06f486f64aa98e71be6b5947f13f"
format_stamp: "Formatted at 2026-08-06 19:47:28 on dist-test-slave-77v9"
server_key: "e1199a806d62f6836b97cfe34f7bcd02"
server_key_iv: "f8f71afbbb67b53607a9baa94fc784a9"
server_key_version: "encryptionkey@0"
I20260806 19:47:28.029170 9529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:28.051012 9529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:28.055902 9529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:28.056253 9529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:28.077030 9529 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:40977
I20260806 19:47:28.077082 9595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:40977 every 8 connection(s)
I20260806 19:47:28.078627 9529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:28.082655 9596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:28.088975 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 9529
I20260806 19:47:28.089211 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:28.089728 22313 external_mini_cluster.cc:1428] Setting key cb33b0aa4748dca941bde5c96551e728
I20260806 19:47:28.094125 9596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f: Bootstrap starting.
I20260806 19:47:28.098021 9596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:28.099251 9596 log.cc:824] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:28.102548 9596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f: No bootstrap required, opened a new log
Aug 06 19:47:28 dist-test-slave-77v9 krb5kdc[9516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:47:28.111328 9596 raft_consensus.cc:348] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37d06f486f64aa98e71be6b5947f13f" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40977 } }
I20260806 19:47:28.111732 9596 raft_consensus.cc:374] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:28.111850 9596 raft_consensus.cc:729] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37d06f486f64aa98e71be6b5947f13f, State: Initialized, Role: FOLLOWER
I20260806 19:47:28.112545 9596 consensus_queue.cc:260] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37d06f486f64aa98e71be6b5947f13f" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40977 } }
I20260806 19:47:28.112826 9596 raft_consensus.cc:388] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:28.112978 9596 raft_consensus.cc:482] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:28.113183 9596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:28.115510 9599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:28.092049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50078 (local address 127.21.202.126:40977)
0806 19:47:28.092980 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:28.092992 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:28.093038 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:47:28.095031 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:28.095057 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:28.095491 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:28.095865 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:28.097956 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:28.099738 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:28.101519 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:28.102247 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:28.102639 (+ 392us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:28.107367 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:28.107379 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:28.107395 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:28.107452 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:28.110801 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:28.111576 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:28.111582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:28.111589 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:28.111866 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:28.112294 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:28.112299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:28.112302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:28.112975 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:28.113300 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:28.113763 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 19:47:28.114247 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":425,"threads_started":1}
I20260806 19:47:28.115882 9596 raft_consensus.cc:504] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37d06f486f64aa98e71be6b5947f13f" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40977 } }
I20260806 19:47:28.116478 9596 leader_election.cc:302] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d37d06f486f64aa98e71be6b5947f13f; no voters:
I20260806 19:47:28.117503 9596 leader_election.cc:288] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:28.117678 9601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:28.119699 9596 sys_catalog.cc:564] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:28.124365 9601 raft_consensus.cc:686] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [term 1 LEADER]: Becoming Leader. State: Replica: d37d06f486f64aa98e71be6b5947f13f, State: Running, Role: LEADER
I20260806 19:47:28.124994 9601 consensus_queue.cc:237] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37d06f486f64aa98e71be6b5947f13f" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40977 } }
I20260806 19:47:28.129885 9602 sys_catalog.cc:455] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d37d06f486f64aa98e71be6b5947f13f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37d06f486f64aa98e71be6b5947f13f" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40977 } } }
I20260806 19:47:28.130259 9603 sys_catalog.cc:455] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d37d06f486f64aa98e71be6b5947f13f. Latest consensus state: current_term: 1 leader_uuid: "d37d06f486f64aa98e71be6b5947f13f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37d06f486f64aa98e71be6b5947f13f" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40977 } } }
I20260806 19:47:28.130568 9603 sys_catalog.cc:458] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:28.130280 9602 sys_catalog.cc:458] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f [sys.catalog]: This master's current role is: LEADER
W20260806 19:47:28.134251 9616 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:47:28.134358 9616 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:47:28.134516 9618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:28.136212 9618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:28.145144 9618 catalog_manager.cc:1347] Generated new cluster ID: 9ac5471015c74a73af3a369b7eb19921
I20260806 19:47:28.145360 9618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:28.161033 9618 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:28.163010 9618 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:28.177908 9618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f: Generated new TSK 0
I20260806 19:47:28.178927 9618 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
Entry for principal kudu/127.21.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab.
I20260806 19:47:28.265129 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:40977
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:28.658654 9623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:28.659194 9623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:28.659389 9623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:28.666934 9623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:28.667127 9623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:28.667323 9623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:47:28.677932 9623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.65.keytab
--principal=kudu/127.21.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:40977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:28.679447 9623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:28.680980 9623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:28.695202 9629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:28.695253 9630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:28.695178 9632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:28.696544 9623 server_base.cc:1034] running on GCE node
I20260806 19:47:28.697386 9623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:28.698484 9623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:28.699751 9623 hybrid_clock.cc:648] HybridClock initialized: now 1786045648699712 us; error 51 us; skew 500 ppm
I20260806 19:47:28.700227 9623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:28 dist-test-slave-77v9 krb5kdc[9516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045648, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:28.703918 9623 init.cc:377] Logged in from keytab as kudu/127.21.202.65@KRBTEST.COM (short username kudu)
I20260806 19:47:28.705605 9623 webserver.cc:460] Webserver started at http://127.21.202.65:39825/ using document root <none> and password file <none>
I20260806 19:47:28.706413 9623 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:28.706539 9623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:28.706825 9623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:28.710103 9623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "b674fe9fd23c4deebfd5ae3c99d90372"
format_stamp: "Formatted at 2026-08-06 19:47:28 on dist-test-slave-77v9"
server_key: "2512c362ddab3a9b14b399baa704381c"
server_key_iv: "727d3b31fb1137f4e3cf967880719b77"
server_key_version: "encryptionkey@0"
I20260806 19:47:28.710866 9623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "b674fe9fd23c4deebfd5ae3c99d90372"
format_stamp: "Formatted at 2026-08-06 19:47:28 on dist-test-slave-77v9"
server_key: "2512c362ddab3a9b14b399baa704381c"
server_key_iv: "727d3b31fb1137f4e3cf967880719b77"
server_key_version: "encryptionkey@0"
I20260806 19:47:28.716333 9623 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:47:28.720640 9639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:28.723060 9623 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:47:28.723235 9623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "b674fe9fd23c4deebfd5ae3c99d90372"
format_stamp: "Formatted at 2026-08-06 19:47:28 on dist-test-slave-77v9"
server_key: "2512c362ddab3a9b14b399baa704381c"
server_key_iv: "727d3b31fb1137f4e3cf967880719b77"
server_key_version: "encryptionkey@0"
I20260806 19:47:28.723470 9623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:28.745743 9623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:28.750775 9623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:28.751062 9623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:28.752045 9623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:28.753474 9623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:28.753548 9623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:28.753636 9623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:28.753680 9623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:28.790086 9623 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:39781
I20260806 19:47:28.790132 9751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:39781 every 8 connection(s)
I20260806 19:47:28.791608 9623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:47:28.801316 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 9623
I20260806 19:47:28.801523 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:47:28.802065 22313 external_mini_cluster.cc:1428] Setting key 0f38e948f78110b13e99b3908d2e1236
Aug 06 19:47:28 dist-test-slave-77v9 krb5kdc[9516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045648, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.65@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:47:28.815090 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:28.794447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:50893 (local address 127.21.202.126:40977)
0806 19:47:28.794964 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:28.794969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:28.795908 (+ 939us) server_negotiation.cc:407] Connection header received
0806 19:47:28.797846 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:28.797852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:28.797942 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:28.798063 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:28.799586 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:28.800401 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:28.802792 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:28.803383 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:28.803544 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:28.807281 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:28.807286 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:28.807289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:28.807324 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:28.810755 (+ 3431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:28.811798 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:28.811806 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:28.811811 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:28.811970 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:28.812520 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:28.812528 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:28.812533 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:28.813121 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:28.813298 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:28.814320 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0806 19:47:28.814610 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":309,"threads_started":1}
I20260806 19:47:28.816102 9755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:28.794863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:40977 (local address 127.21.202.65:50893)
0806 19:47:28.795732 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:28.795782 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:28.797568 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:28.798307 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:28.798315 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:28.798840 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:28.799408 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:28.799426 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:28.800575 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:28.800579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:28.802586 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:28.802604 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:28.803644 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:28.803652 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:28.803892 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:28.804710 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:47:28.804738 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:28.807026 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:28.810921 (+ 3895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:28.810933 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:28.810954 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:28.811652 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:28.812084 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:28.812091 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:28.812098 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:28.812370 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:28.813291 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:28.813303 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:28.813794 (+ 491us) client_negotiation.cc:769] Sending connection context
0806 19:47:28.814136 (+ 342us) client_negotiation.cc:240] Negotiation successful
0806 19:47:28.814699 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":383,"threads_started":1}
I20260806 19:47:28.818137 9752 heartbeater.cc:347] Connected to a master server at 127.21.202.126:40977
I20260806 19:47:28.818488 9752 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:28.819406 9752 heartbeater.cc:511] Master 127.21.202.126:40977 requested a full tablet report, sending...
I20260806 19:47:28.821899 9561 ts_manager.cc:194] Registered new tserver with Master: b674fe9fd23c4deebfd5ae3c99d90372 (127.21.202.65:39781)
I20260806 19:47:28.823414 9561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.65@KRBTEST.COM'} at 127.21.202.65:50893
WARNING: no policy specified for kudu/127.21.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
Entry for principal kudu/127.21.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab.
I20260806 19:47:28.877245 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:40977
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:29.244649 9758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:29.244982 9758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:29.245174 9758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:29.252701 9758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:29.252847 9758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:29.253033 9758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:47:29.263161 9758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.66.keytab
--principal=kudu/127.21.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:40977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:29.264554 9758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:29.266017 9758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:29.279376 9763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:29.279657 9766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:29.279686 9764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:29.281248 9758 server_base.cc:1034] running on GCE node
I20260806 19:47:29.281934 9758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:29.282977 9758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:29.284240 9758 hybrid_clock.cc:648] HybridClock initialized: now 1786045649284197 us; error 38 us; skew 500 ppm
I20260806 19:47:29.284608 9758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:29 dist-test-slave-77v9 krb5kdc[9516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045649, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:29.288237 9758 init.cc:377] Logged in from keytab as kudu/127.21.202.66@KRBTEST.COM (short username kudu)
I20260806 19:47:29.290045 9758 webserver.cc:460] Webserver started at http://127.21.202.66:37227/ using document root <none> and password file <none>
I20260806 19:47:29.290808 9758 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:29.290921 9758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:29.291359 9758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:29.294552 9758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "4580076fdcbe4f50b5e8ceb11502f2bc"
format_stamp: "Formatted at 2026-08-06 19:47:29 on dist-test-slave-77v9"
server_key: "e6d105921377df3ca1237bb8d3e76867"
server_key_iv: "f7d9faacba011f3a84f18da2e1f408ea"
server_key_version: "encryptionkey@0"
I20260806 19:47:29.295320 9758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "4580076fdcbe4f50b5e8ceb11502f2bc"
format_stamp: "Formatted at 2026-08-06 19:47:29 on dist-test-slave-77v9"
server_key: "e6d105921377df3ca1237bb8d3e76867"
server_key_iv: "f7d9faacba011f3a84f18da2e1f408ea"
server_key_version: "encryptionkey@0"
I20260806 19:47:29.301226 9758 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 19:47:29.305311 9773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:29.306984 9758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:47:29.307132 9758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "4580076fdcbe4f50b5e8ceb11502f2bc"
format_stamp: "Formatted at 2026-08-06 19:47:29 on dist-test-slave-77v9"
server_key: "e6d105921377df3ca1237bb8d3e76867"
server_key_iv: "f7d9faacba011f3a84f18da2e1f408ea"
server_key_version: "encryptionkey@0"
I20260806 19:47:29.307303 9758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:29.321668 9758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:29.326292 9758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:29.326586 9758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:29.327808 9758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:29.329366 9758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:29.329450 9758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:29.329545 9758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:29.329593 9758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:29.366679 9758 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:45873
I20260806 19:47:29.366730 9887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:45873 every 8 connection(s)
I20260806 19:47:29.368048 9758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:47:29.372875 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 9758
I20260806 19:47:29.373082 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:47:29.373564 22313 external_mini_cluster.cc:1428] Setting key ccfb2fb8395df5168b095192f9cd424d
Aug 06 19:47:29 dist-test-slave-77v9 krb5kdc[9516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045649, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.66@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:47:29.390899 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:29.370900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:54191 (local address 127.21.202.126:40977)
0806 19:47:29.371372 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:29.371376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:29.372425 (+ 1049us) server_negotiation.cc:407] Connection header received
0806 19:47:29.374682 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:29.374688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:29.374786 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:29.374913 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:29.376947 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:29.377750 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:29.379407 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:29.379840 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:29.379979 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:29.383825 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:29.383830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:29.383833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:29.383913 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:29.387088 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:29.388215 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:29.388220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:29.388222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:29.388377 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:29.389014 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:29.389018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:29.389020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:29.389297 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:29.389415 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:29.390432 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0806 19:47:29.390614 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":351,"thread_start_us":239,"threads_started":1}
I20260806 19:47:29.392604 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:29.371463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:40977 (local address 127.21.202.66:54191)
0806 19:47:29.372226 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:29.372271 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:29.374393 (+ 2122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:29.375176 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:29.375185 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:29.375906 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:29.376621 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:29.376645 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:29.377951 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:29.377955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:29.379237 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:29.379250 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:29.380095 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:29.380103 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:29.380346 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:29.381186 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:47:29.381211 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:29.383518 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:29.387291 (+ 3773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:29.387311 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:29.387328 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:29.388038 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:29.388522 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:29.388531 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:29.388534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:29.388865 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:29.389440 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:29.389452 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:29.389968 (+ 516us) client_negotiation.cc:769] Sending connection context
0806 19:47:29.390318 (+ 350us) client_negotiation.cc:240] Negotiation successful
0806 19:47:29.390964 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":331,"threads_started":1}
I20260806 19:47:29.394256 9888 heartbeater.cc:347] Connected to a master server at 127.21.202.126:40977
I20260806 19:47:29.394618 9888 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:29.395577 9888 heartbeater.cc:511] Master 127.21.202.126:40977 requested a full tablet report, sending...
I20260806 19:47:29.397527 9561 ts_manager.cc:194] Registered new tserver with Master: 4580076fdcbe4f50b5e8ceb11502f2bc (127.21.202.66:45873)
I20260806 19:47:29.398962 9561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.66@KRBTEST.COM'} at 127.21.202.66:54191
WARNING: no policy specified for kudu/127.21.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
Entry for principal kudu/127.21.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab.
I20260806 19:47:29.457624 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:40977
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:47:29.827446 9752 heartbeater.cc:503] Master 127.21.202.126:40977 was elected leader, sending a full tablet report...
W20260806 19:47:29.844071 9894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:29.844450 9894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:29.844641 9894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:29.853271 9894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:29.853457 9894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:29.853675 9894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:47:29.865262 9894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.67.keytab
--principal=kudu/127.21.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:40977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:29.867017 9894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:29.869215 9894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:29.885401 9899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:29.885383 9902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:29.885368 9900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:29.886739 9894 server_base.cc:1034] running on GCE node
I20260806 19:47:29.887475 9894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:29.888599 9894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:29.889838 9894 hybrid_clock.cc:648] HybridClock initialized: now 1786045649889751 us; error 66 us; skew 500 ppm
I20260806 19:47:29.890221 9894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:29 dist-test-slave-77v9 krb5kdc[9516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045649, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:29.893748 9894 init.cc:377] Logged in from keytab as kudu/127.21.202.67@KRBTEST.COM (short username kudu)
I20260806 19:47:29.895404 9894 webserver.cc:460] Webserver started at http://127.21.202.67:45133/ using document root <none> and password file <none>
I20260806 19:47:29.896080 9894 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:29.896183 9894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:29.896453 9894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:29.899492 9894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "ab993576f87e4a16b1197d538df407d2"
format_stamp: "Formatted at 2026-08-06 19:47:29 on dist-test-slave-77v9"
server_key: "8dcae4bc96bc4b9ca427c25baa09bd05"
server_key_iv: "aedc706acb1e6982fbd290d9d971de6e"
server_key_version: "encryptionkey@0"
I20260806 19:47:29.900239 9894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "ab993576f87e4a16b1197d538df407d2"
format_stamp: "Formatted at 2026-08-06 19:47:29 on dist-test-slave-77v9"
server_key: "8dcae4bc96bc4b9ca427c25baa09bd05"
server_key_iv: "aedc706acb1e6982fbd290d9d971de6e"
server_key_version: "encryptionkey@0"
I20260806 19:47:29.905810 9894 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 19:47:29.909938 9909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:29.911588 9894 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:47:29.911792 9894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "ab993576f87e4a16b1197d538df407d2"
format_stamp: "Formatted at 2026-08-06 19:47:29 on dist-test-slave-77v9"
server_key: "8dcae4bc96bc4b9ca427c25baa09bd05"
server_key_iv: "aedc706acb1e6982fbd290d9d971de6e"
server_key_version: "encryptionkey@0"
I20260806 19:47:29.912025 9894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:29.936297 9894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:29.941449 9894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:29.941798 9894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:29.942937 9894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:29.944476 9894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:29.944593 9894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:29.944757 9894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:29.944865 9894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:29.982076 9894 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:36503
I20260806 19:47:29.982126 10023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:36503 every 8 connection(s)
I20260806 19:47:29.983487 9894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:47:29.993835 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 9894
I20260806 19:47:29.994050 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:47:29.994560 22313 external_mini_cluster.cc:1428] Setting key a7e0ce96bc9661b68e0de8718023972f
Aug 06 19:47:29 dist-test-slave-77v9 krb5kdc[9516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045649, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.67@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:47:30.005848 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:29.986523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:60225 (local address 127.21.202.126:40977)
0806 19:47:29.987045 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:29.987051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:29.988150 (+ 1099us) server_negotiation.cc:407] Connection header received
0806 19:47:29.990278 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:29.990287 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:29.990393 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:29.990529 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:29.992199 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:29.993148 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:29.995123 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:29.995551 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:29.995657 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:29.999916 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:29.999921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:29.999924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:29.999962 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:30.002670 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:30.003608 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:30.003612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:30.003615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:30.003740 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:30.004205 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:30.004208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:30.004210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:30.004493 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:30.004614 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:30.005404 (+ 790us) server_negotiation.cc:299] Negotiation successful
0806 19:47:30.005578 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":230,"threads_started":1}
I20260806 19:47:30.007014 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:29.987029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:40977 (local address 127.21.202.67:60225)
0806 19:47:29.987893 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:29.987943 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:29.989964 (+ 2021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:29.990787 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:29.990796 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:29.991399 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:29.992003 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:29.992025 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:29.993331 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:29.993344 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:29.994919 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:29.994932 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:29.995817 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:29.995829 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:29.996170 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:29.997080 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:47:29.997109 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:29.999676 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:30.002821 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:30.002829 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:30.002845 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:30.003472 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:30.003892 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:47:30.003896 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:47:30.003898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:47:30.004099 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:47:30.004603 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:30.004609 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:30.005046 (+ 437us) client_negotiation.cc:769] Sending connection context
0806 19:47:30.005294 (+ 248us) client_negotiation.cc:240] Negotiation successful
0806 19:47:30.005712 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":329,"threads_started":1}
I20260806 19:47:30.008876 10024 heartbeater.cc:347] Connected to a master server at 127.21.202.126:40977
I20260806 19:47:30.009294 10024 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:30.010232 10024 heartbeater.cc:511] Master 127.21.202.126:40977 requested a full tablet report, sending...
I20260806 19:47:30.012059 9561 ts_manager.cc:194] Registered new tserver with Master: ab993576f87e4a16b1197d538df407d2 (127.21.202.67:36503)
I20260806 19:47:30.013027 9561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.202.67@KRBTEST.COM'} at 127.21.202.67:60225
I20260806 19:47:30.023139 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:47:30.042371 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.028168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55492 (local address 127.21.202.126:40977)
0806 19:47:30.028401 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:30.028427 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:30.028969 (+ 542us) server_negotiation.cc:407] Connection header received
0806 19:47:30.029198 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:30.029211 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:30.029304 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:30.029400 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:30.030875 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.031622 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.033243 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.033690 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.033805 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.036258 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:30.036262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:30.036266 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:30.036301 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:30.039170 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:30.039980 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:30.039985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:30.039988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:30.040150 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:30.040625 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:30.040629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:30.040631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:30.041188 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:30.041418 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:30.041732 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 19:47:30.041945 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 19:47:30.046937 9561 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:47:30.054350 9561 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:47:30.083985 10039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.070087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52484 (local address 127.21.202.65:39781)
0806 19:47:30.070928 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:30.070942 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:30.072279 (+ 1337us) server_negotiation.cc:407] Connection header received
0806 19:47:30.075206 (+ 2927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:30.075213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:30.075453 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:30.075638 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:30.076687 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.078025 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.080762 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.082339 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.082458 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.082698 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:30.083425 (+ 727us) server_negotiation.cc:299] Negotiation successful
0806 19:47:30.083622 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":440,"threads_started":1}
I20260806 19:47:30.085016 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.070025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33536 (local address 127.21.202.66:45873)
0806 19:47:30.071397 (+ 1372us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:30.071409 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:30.073121 (+ 1712us) server_negotiation.cc:407] Connection header received
0806 19:47:30.073666 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:30.073679 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:30.073999 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:30.074192 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:30.074880 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.079235 (+ 4355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.082080 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.083559 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.083689 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.083870 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:30.084462 (+ 592us) server_negotiation.cc:299] Negotiation successful
0806 19:47:30.084641 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1192,"thread_start_us":1066,"threads_started":1}
I20260806 19:47:30.085431 10041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.069549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:39781 (local address 127.0.0.1:52484)
0806 19:47:30.072076 (+ 2527us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:30.072124 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:30.074539 (+ 2415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:30.075970 (+ 1431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:30.075978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:30.076010 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:30.076484 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.076498 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.078189 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.078192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.080547 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.080564 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.082901 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.082913 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.083141 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.083198 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 19:47:30.084490 (+ 1292us) client_negotiation.cc:240] Negotiation successful
0806 19:47:30.084738 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2214,"mutex_wait_us":27,"thread_start_us":4304,"threads_started":1}
I20260806 19:47:30.086591 10042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.070500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36694 (local address 127.21.202.67:36503)
0806 19:47:30.075049 (+ 4549us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:30.075056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:30.075113 (+ 57us) server_negotiation.cc:407] Connection header received
0806 19:47:30.075200 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:30.075206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:30.075437 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:30.075638 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:30.076381 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.078972 (+ 2591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.083171 (+ 4199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.084693 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.084857 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.085098 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:30.086002 (+ 904us) server_negotiation.cc:299] Negotiation successful
0806 19:47:30.086194 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4389,"thread_start_us":4207,"threads_started":1}
I20260806 19:47:30.085886 10037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.069589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:45873 (local address 127.0.0.1:33536)
0806 19:47:30.072966 (+ 3377us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:30.072990 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:30.073357 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:30.074201 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:30.074210 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:30.074239 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:30.074682 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.074696 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.079410 (+ 4714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.079416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.081867 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.081882 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.084064 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.084070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.084247 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.084280 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 19:47:30.085278 (+ 998us) client_negotiation.cc:240] Negotiation successful
0806 19:47:30.085460 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3190,"mutex_wait_us":61,"spinlock_wait_cycles":5248,"thread_start_us":2010,"threads_started":1}
I20260806 19:47:30.087539 10040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.069801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:36503 (local address 127.0.0.1:36694)
0806 19:47:30.074211 (+ 4410us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:30.074234 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:30.074389 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:30.075657 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:30.075662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:30.075692 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:30.076122 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.076136 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.079135 (+ 2999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.079140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.082970 (+ 3830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.082982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.085247 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.085255 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.085498 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.085545 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 19:47:30.085640 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 19:47:30.085793 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4252,"mutex_wait_us":146,"thread_start_us":2781,"threads_started":1}
I20260806 19:47:30.090615 9821 tablet_service.cc:1467] Processing CreateTablet for tablet 533ee666e6474a77b16e8d89987f0cd4 (DEFAULT_TABLE table=test-table [id=255e5692049440b4aff6c6710813a508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:30.090636 9687 tablet_service.cc:1467] Processing CreateTablet for tablet 533ee666e6474a77b16e8d89987f0cd4 (DEFAULT_TABLE table=test-table [id=255e5692049440b4aff6c6710813a508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:30.092124 9687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 533ee666e6474a77b16e8d89987f0cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:30.092751 9821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 533ee666e6474a77b16e8d89987f0cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:30.094151 9958 tablet_service.cc:1467] Processing CreateTablet for tablet 533ee666e6474a77b16e8d89987f0cd4 (DEFAULT_TABLE table=test-table [id=255e5692049440b4aff6c6710813a508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:30.097329 9958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 533ee666e6474a77b16e8d89987f0cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:30.108455 10043 tablet_bootstrap.cc:492] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc: Bootstrap starting.
I20260806 19:47:30.110591 10044 tablet_bootstrap.cc:492] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372: Bootstrap starting.
I20260806 19:47:30.112912 10043 tablet_bootstrap.cc:654] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:30.114547 10043 log.cc:824] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:30.114804 10044 tablet_bootstrap.cc:654] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:30.116356 10044 log.cc:824] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:30.118083 10043 tablet_bootstrap.cc:492] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc: No bootstrap required, opened a new log
I20260806 19:47:30.118422 10043 ts_tablet_manager.cc:1397] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 19:47:30.119192 10044 tablet_bootstrap.cc:492] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372: No bootstrap required, opened a new log
I20260806 19:47:30.119457 10044 ts_tablet_manager.cc:1397] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260806 19:47:30.120944 10045 tablet_bootstrap.cc:492] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2: Bootstrap starting.
I20260806 19:47:30.125073 10045 tablet_bootstrap.cc:654] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:30.126534 10045 log.cc:824] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:30.128260 10044 raft_consensus.cc:348] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39781 } } peers { permanent_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45873 } } peers { permanent_uuid: "ab993576f87e4a16b1197d538df407d2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36503 } }
I20260806 19:47:30.128753 10044 raft_consensus.cc:374] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:30.128885 10044 raft_consensus.cc:729] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b674fe9fd23c4deebfd5ae3c99d90372, State: Initialized, Role: FOLLOWER
I20260806 19:47:30.129141 10043 raft_consensus.cc:348] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39781 } } peers { permanent_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45873 } } peers { permanent_uuid: "ab993576f87e4a16b1197d538df407d2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36503 } }
I20260806 19:47:30.129652 10043 raft_consensus.cc:374] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:30.129545 10044 consensus_queue.cc:260] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39781 } } peers { permanent_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45873 } } peers { permanent_uuid: "ab993576f87e4a16b1197d538df407d2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36503 } }
I20260806 19:47:30.129766 10043 raft_consensus.cc:729] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4580076fdcbe4f50b5e8ceb11502f2bc, State: Initialized, Role: FOLLOWER
I20260806 19:47:30.130386 10043 consensus_queue.cc:260] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39781 } } peers { permanent_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45873 } } peers { permanent_uuid: "ab993576f87e4a16b1197d538df407d2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36503 } }
I20260806 19:47:30.130591 10045 tablet_bootstrap.cc:492] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2: No bootstrap required, opened a new log
I20260806 19:47:30.130918 10045 ts_tablet_manager.cc:1397] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.006s
I20260806 19:47:30.131682 9888 heartbeater.cc:503] Master 127.21.202.126:40977 was elected leader, sending a full tablet report...
I20260806 19:47:30.132306 10043 ts_tablet_manager.cc:1428] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260806 19:47:30.134752 10044 ts_tablet_manager.cc:1428] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372: Time spent starting tablet: real 0.015s user 0.006s sys 0.007s
I20260806 19:47:30.140643 10045 raft_consensus.cc:348] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39781 } } peers { permanent_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45873 } } peers { permanent_uuid: "ab993576f87e4a16b1197d538df407d2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36503 } }
I20260806 19:47:30.141124 10045 raft_consensus.cc:374] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:30.141233 10045 raft_consensus.cc:729] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab993576f87e4a16b1197d538df407d2, State: Initialized, Role: FOLLOWER
I20260806 19:47:30.141830 10045 consensus_queue.cc:260] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39781 } } peers { permanent_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45873 } } peers { permanent_uuid: "ab993576f87e4a16b1197d538df407d2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36503 } }
I20260806 19:47:30.142865 10024 heartbeater.cc:503] Master 127.21.202.126:40977 was elected leader, sending a full tablet report...
I20260806 19:47:30.143602 10045 ts_tablet_manager.cc:1428] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2: Time spent starting tablet: real 0.012s user 0.009s sys 0.002s
I20260806 19:47:30.162463 10051 raft_consensus.cc:482] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:47:30.162858 10051 raft_consensus.cc:504] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39781 } } peers { permanent_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45873 } } peers { permanent_uuid: "ab993576f87e4a16b1197d538df407d2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36503 } }
I20260806 19:47:30.164922 10051 leader_election.cc:288] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b674fe9fd23c4deebfd5ae3c99d90372 (127.21.202.65:39781), 4580076fdcbe4f50b5e8ceb11502f2bc (127.21.202.66:45873)
I20260806 19:47:30.167364 10050 raft_consensus.cc:482] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:47:30.167757 10050 raft_consensus.cc:504] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39781 } } peers { permanent_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45873 } } peers { permanent_uuid: "ab993576f87e4a16b1197d538df407d2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36503 } }
I20260806 19:47:30.173441 10039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.165330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:40571 (local address 127.21.202.65:39781)
0806 19:47:30.165608 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:30.165616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:30.165643 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:47:30.165941 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:30.165951 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:30.166044 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:30.166162 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:30.166657 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.167863 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.169831 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.171123 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.171307 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.171424 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:30.172906 (+ 1482us) server_negotiation.cc:299] Negotiation successful
0806 19:47:30.173106 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260806 19:47:30.173894 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.165185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:39781 (local address 127.21.202.67:40571)
0806 19:47:30.165441 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:30.165470 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:30.165648 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:30.166194 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:30.166201 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:30.166240 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:30.166493 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.166506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.168019 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.168024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.169654 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.169666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.171884 (+ 2218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.171890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.172137 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.172192 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 19:47:30.172296 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 19:47:30.172490 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":1}
I20260806 19:47:30.176373 10050 leader_election.cc:288] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b674fe9fd23c4deebfd5ae3c99d90372 (127.21.202.65:39781), ab993576f87e4a16b1197d538df407d2 (127.21.202.67:36503)
I20260806 19:47:30.178437 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.165782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:40191 (local address 127.21.202.66:45873)
0806 19:47:30.166051 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:30.166057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:30.166365 (+ 308us) server_negotiation.cc:407] Connection header received
0806 19:47:30.166534 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:30.166539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:30.166667 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:30.166836 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:30.167436 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.168560 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.171983 (+ 3423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.176734 (+ 4751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.176927 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.177012 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:30.177834 (+ 822us) server_negotiation.cc:299] Negotiation successful
0806 19:47:30.178038 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 19:47:30.179421 10052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.165320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:45873 (local address 127.21.202.67:40191)
0806 19:47:30.166218 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:30.166240 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:30.166376 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:30.166851 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:30.166854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:30.166875 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:30.167254 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.167264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.168750 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.168758 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.171683 (+ 2925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.171711 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.177270 (+ 5559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.177281 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.177563 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.177635 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 19:47:30.178633 (+ 998us) client_negotiation.cc:240] Negotiation successful
0806 19:47:30.178844 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":65,"thread_start_us":520,"threads_started":1}
I20260806 19:47:30.180742 9707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "533ee666e6474a77b16e8d89987f0cd4" candidate_uuid: "ab993576f87e4a16b1197d538df407d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" is_pre_election: true
I20260806 19:47:30.180713 9842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "533ee666e6474a77b16e8d89987f0cd4" candidate_uuid: "ab993576f87e4a16b1197d538df407d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" is_pre_election: true
I20260806 19:47:30.181391 9842 raft_consensus.cc:2454] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab993576f87e4a16b1197d538df407d2 in term 0.
I20260806 19:47:30.181430 9707 raft_consensus.cc:2454] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab993576f87e4a16b1197d538df407d2 in term 0.
I20260806 19:47:30.182216 9912 leader_election.cc:302] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab993576f87e4a16b1197d538df407d2, b674fe9fd23c4deebfd5ae3c99d90372; no voters:
I20260806 19:47:30.182747 10051 raft_consensus.cc:2781] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:47:30.182955 10051 raft_consensus.cc:482] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:47:30.183082 10051 raft_consensus.cc:3037] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:30.183835 10039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.170748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:54841 (local address 127.21.202.65:39781)
0806 19:47:30.173728 (+ 2980us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:30.173737 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:30.174995 (+ 1258us) server_negotiation.cc:407] Connection header received
0806 19:47:30.175300 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:30.175308 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:30.175407 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:30.175482 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:30.176589 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.178190 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.181371 (+ 3181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.182667 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.182830 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.182925 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:30.183345 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 19:47:30.183510 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2850,"thread_start_us":9083,"threads_started":1}
I20260806 19:47:30.184546 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.170359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:39781 (local address 127.21.202.66:54841)
0806 19:47:30.174874 (+ 4515us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:30.174900 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:30.175129 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:30.175715 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:30.175723 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:30.175756 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:30.176374 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.176388 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.178425 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.178434 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.181181 (+ 2747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.181196 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.182916 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.182922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.183136 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.183182 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 19:47:30.183978 (+ 796us) client_negotiation.cc:240] Negotiation successful
0806 19:47:30.184148 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4316,"thread_start_us":4101,"threads_started":1}
I20260806 19:47:30.185405 9707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "533ee666e6474a77b16e8d89987f0cd4" candidate_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" is_pre_election: true
I20260806 19:47:30.185654 9707 raft_consensus.cc:2454] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4580076fdcbe4f50b5e8ceb11502f2bc in term 0.
I20260806 19:47:30.186141 9776 leader_election.cc:302] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4580076fdcbe4f50b5e8ceb11502f2bc, b674fe9fd23c4deebfd5ae3c99d90372; no voters:
I20260806 19:47:30.186537 10050 raft_consensus.cc:2781] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:47:30.186697 10050 raft_consensus.cc:482] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:47:30.186827 10050 raft_consensus.cc:3037] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:30.186817 10051 raft_consensus.cc:504] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39781 } } peers { permanent_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45873 } } peers { permanent_uuid: "ab993576f87e4a16b1197d538df407d2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36503 } }
I20260806 19:47:30.187641 10051 leader_election.cc:288] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [CANDIDATE]: Term 1 election: Requested vote from peers b674fe9fd23c4deebfd5ae3c99d90372 (127.21.202.65:39781), 4580076fdcbe4f50b5e8ceb11502f2bc (127.21.202.66:45873)
I20260806 19:47:30.188202 10042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.171741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:37707 (local address 127.21.202.67:36503)
0806 19:47:30.171919 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:30.171925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:30.180689 (+ 8764us) server_negotiation.cc:407] Connection header received
0806 19:47:30.181153 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:30.181158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:30.181269 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:30.181341 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:30.182211 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.183359 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.185781 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.187150 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.187314 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.187402 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:30.187815 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 19:47:30.187960 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 19:47:30.188159 9707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "533ee666e6474a77b16e8d89987f0cd4" candidate_uuid: "ab993576f87e4a16b1197d538df407d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b674fe9fd23c4deebfd5ae3c99d90372"
I20260806 19:47:30.188196 9842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "533ee666e6474a77b16e8d89987f0cd4" candidate_uuid: "ab993576f87e4a16b1197d538df407d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc"
I20260806 19:47:30.188390 9707 raft_consensus.cc:3037] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:30.188925 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.170524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:36503 (local address 127.21.202.66:37707)
0806 19:47:30.180583 (+ 10059us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:30.180604 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:30.180798 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:30.181564 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:30.181568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:30.181607 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:30.182043 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.182054 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.183528 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.183533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.185603 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:30.185618 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.187410 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.187418 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:30.187586 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.187621 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 19:47:30.188358 (+ 737us) client_negotiation.cc:240] Negotiation successful
0806 19:47:30.188508 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9920,"thread_start_us":11937,"threads_started":1}
I20260806 19:47:30.190002 9978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "533ee666e6474a77b16e8d89987f0cd4" candidate_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab993576f87e4a16b1197d538df407d2" is_pre_election: true
I20260806 19:47:30.190383 10050 raft_consensus.cc:504] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39781 } } peers { permanent_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45873 } } peers { permanent_uuid: "ab993576f87e4a16b1197d538df407d2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36503 } }
I20260806 19:47:30.190517 9978 raft_consensus.cc:2379] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4580076fdcbe4f50b5e8ceb11502f2bc in current term 1: Already voted for candidate ab993576f87e4a16b1197d538df407d2 in this term.
I20260806 19:47:30.190778 9842 raft_consensus.cc:2379] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ab993576f87e4a16b1197d538df407d2 in current term 1: Already voted for candidate 4580076fdcbe4f50b5e8ceb11502f2bc in this term.
I20260806 19:47:30.191107 10050 leader_election.cc:288] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [CANDIDATE]: Term 1 election: Requested vote from peers b674fe9fd23c4deebfd5ae3c99d90372 (127.21.202.65:39781), ab993576f87e4a16b1197d538df407d2 (127.21.202.67:36503)
I20260806 19:47:30.191619 9707 raft_consensus.cc:2454] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab993576f87e4a16b1197d538df407d2 in term 1.
I20260806 19:47:30.192040 9912 leader_election.cc:302] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab993576f87e4a16b1197d538df407d2, b674fe9fd23c4deebfd5ae3c99d90372; no voters:
I20260806 19:47:30.192514 9978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "533ee666e6474a77b16e8d89987f0cd4" candidate_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab993576f87e4a16b1197d538df407d2"
I20260806 19:47:30.192521 9707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "533ee666e6474a77b16e8d89987f0cd4" candidate_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b674fe9fd23c4deebfd5ae3c99d90372"
I20260806 19:47:30.192761 9978 raft_consensus.cc:2379] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4580076fdcbe4f50b5e8ceb11502f2bc in current term 1: Already voted for candidate ab993576f87e4a16b1197d538df407d2 in this term.
I20260806 19:47:30.192875 9707 raft_consensus.cc:2379] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4580076fdcbe4f50b5e8ceb11502f2bc in current term 1: Already voted for candidate ab993576f87e4a16b1197d538df407d2 in this term.
I20260806 19:47:30.193266 10051 raft_consensus.cc:2781] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:30.193473 9776 leader_election.cc:302] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4580076fdcbe4f50b5e8ceb11502f2bc; no voters: ab993576f87e4a16b1197d538df407d2, b674fe9fd23c4deebfd5ae3c99d90372
I20260806 19:47:30.193766 10050 raft_consensus.cc:2726] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 19:47:30.194000 10051 raft_consensus.cc:686] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 1 LEADER]: Becoming Leader. State: Replica: ab993576f87e4a16b1197d538df407d2, State: Running, Role: LEADER
I20260806 19:47:30.194489 10051 consensus_queue.cc:237] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39781 } } peers { permanent_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45873 } } peers { permanent_uuid: "ab993576f87e4a16b1197d538df407d2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36503 } }
I20260806 19:47:30.199486 9560 catalog_manager.cc:5495] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ab993576f87e4a16b1197d538df407d2 (127.21.202.67). New cstate: current_term: 1 leader_uuid: "ab993576f87e4a16b1197d538df407d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab993576f87e4a16b1197d538df407d2" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 36503 } health_report { overall_health: HEALTHY } } }
W20260806 19:47:30.235527 10025 tablet.cc:2369] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:47:30.243216 10042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.237219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36702 (local address 127.21.202.67:36503)
0806 19:47:30.237454 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:30.237461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:30.237577 (+ 116us) server_negotiation.cc:407] Connection header received
0806 19:47:30.237857 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:30.237861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:30.237942 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:30.238093 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:47:30.238490 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.239298 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.240902 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.241290 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.241423 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.241933 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:47:30.242443 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:47:30.242742 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:30.242808 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 19:47:30.242902 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 19:47:30.252163 9707 raft_consensus.cc:1261] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372 [term 1 FOLLOWER]: Refusing update from remote peer ab993576f87e4a16b1197d538df407d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:47:30.252177 9842 raft_consensus.cc:1261] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 1 FOLLOWER]: Refusing update from remote peer ab993576f87e4a16b1197d538df407d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:47:30.253175 10056 consensus_queue.cc:1035] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4580076fdcbe4f50b5e8ceb11502f2bc" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 45873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:47:30.253619 10051 consensus_queue.cc:1035] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b674fe9fd23c4deebfd5ae3c99d90372" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 39781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:47:30.267299 10059 mvcc.cc:205] Tried to move back new op lower bound from 7315642983420768256 to 7315642983200403456. Current Snapshot: MvccSnapshot[applied={T|T < 7315642983420768256}]
I20260806 19:47:30.268452 10060 mvcc.cc:205] Tried to move back new op lower bound from 7315642983420768256 to 7315642983200403456. Current Snapshot: MvccSnapshot[applied={T|T < 7315642983420768256}]
I20260806 19:47:30.281464 9560 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:47:30.281783 9560 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:47:30.286079 9560 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d37d06f486f64aa98e71be6b5947f13f: Sending DeleteTablet for 3 replicas of tablet 533ee666e6474a77b16e8d89987f0cd4
I20260806 19:47:30.287482 9687 tablet_service.cc:1514] Processing DeleteTablet for tablet 533ee666e6474a77b16e8d89987f0cd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:30 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:52484
I20260806 19:47:30.287581 9958 tablet_service.cc:1514] Processing DeleteTablet for tablet 533ee666e6474a77b16e8d89987f0cd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:30 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:36694
I20260806 19:47:30.288434 9821 tablet_service.cc:1514] Processing DeleteTablet for tablet 533ee666e6474a77b16e8d89987f0cd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:30 UTC) from {username='kudu', principal='kudu/127.21.202.126@KRBTEST.COM'} at 127.0.0.1:33536
I20260806 19:47:30.288843 10070 tablet_replica.cc:330] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2: stopping tablet replica
I20260806 19:47:30.289358 10070 raft_consensus.cc:2229] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:47:30.289680 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 9623
I20260806 19:47:30.290140 10070 raft_consensus.cc:2258] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:30.291092 10069 tablet_replica.cc:330] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372: stopping tablet replica
I20260806 19:47:30.291569 10069 raft_consensus.cc:2229] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:47:30.292125 10069 raft_consensus.cc:2258] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:30.294832 10071 tablet_replica.cc:330] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc: stopping tablet replica
I20260806 19:47:30.295370 10071 raft_consensus.cc:2229] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:47:30.295987 10071 raft_consensus.cc:2258] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:30.296640 10069 ts_tablet_manager.cc:1905] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:30.298985 10070 ts_tablet_manager.cc:1905] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:30.299512 10071 ts_tablet_manager.cc:1905] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:30.304797 10069 ts_tablet_manager.cc:1918] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:30.305085 10069 log.cc:1196] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/533ee666e6474a77b16e8d89987f0cd4
I20260806 19:47:30.305751 10069 ts_tablet_manager.cc:1939] T 533ee666e6474a77b16e8d89987f0cd4 P b674fe9fd23c4deebfd5ae3c99d90372: Deleting consensus metadata
I20260806 19:47:30.306965 10070 ts_tablet_manager.cc:1918] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:30.307238 10070 log.cc:1196] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/533ee666e6474a77b16e8d89987f0cd4
I20260806 19:47:30.307902 10070 ts_tablet_manager.cc:1939] T 533ee666e6474a77b16e8d89987f0cd4 P ab993576f87e4a16b1197d538df407d2: Deleting consensus metadata
I20260806 19:47:30.307981 9547 catalog_manager.cc:4841] TS b674fe9fd23c4deebfd5ae3c99d90372 (127.21.202.65:39781): tablet 533ee666e6474a77b16e8d89987f0cd4 (table test-table [id=255e5692049440b4aff6c6710813a508]) successfully deleted
I20260806 19:47:30.309791 9546 catalog_manager.cc:4841] TS ab993576f87e4a16b1197d538df407d2 (127.21.202.67:36503): tablet 533ee666e6474a77b16e8d89987f0cd4 (table test-table [id=255e5692049440b4aff6c6710813a508]) successfully deleted
I20260806 19:47:30.311304 10071 ts_tablet_manager.cc:1918] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:30.311510 10071 log.cc:1196] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/533ee666e6474a77b16e8d89987f0cd4
I20260806 19:47:30.312111 10071 ts_tablet_manager.cc:1939] T 533ee666e6474a77b16e8d89987f0cd4 P 4580076fdcbe4f50b5e8ceb11502f2bc: Deleting consensus metadata
I20260806 19:47:30.314229 9548 catalog_manager.cc:4841] TS 4580076fdcbe4f50b5e8ceb11502f2bc (127.21.202.66:45873): tablet 533ee666e6474a77b16e8d89987f0cd4 (table test-table [id=255e5692049440b4aff6c6710813a508]) successfully deleted
Aug 06 19:47:30 dist-test-slave-77v9 krb5kdc[9516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.65@KRBTEST.COM
I20260806 19:47:30.325614 10039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.292127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52494 (local address 127.21.202.65:39781)
0806 19:47:30.292323 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:30.292330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:30.294430 (+ 2100us) server_negotiation.cc:407] Connection header received
0806 19:47:30.295193 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:30.295199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:30.295322 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:30.295427 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:30.299783 (+ 4356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.300930 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.307811 (+ 6881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.308498 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.308633 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.319039 (+ 10406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:30.319059 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:30.319076 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:30.319123 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:30.322390 (+ 3267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:30.323257 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:30.323265 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:30.323267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:30.323415 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:30.323831 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:30.323835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:30.323837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:30.324218 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:30.324570 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:30.324984 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 19:47:30.325247 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 19:47:30.326505 9747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:30.437045 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 9623
I20260806 19:47:30.465055 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 9758
Aug 06 19:47:30 dist-test-slave-77v9 krb5kdc[9516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.66@KRBTEST.COM
I20260806 19:47:30.485306 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.466084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33540 (local address 127.21.202.66:45873)
0806 19:47:30.466290 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:30.466297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:30.466319 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:47:30.466400 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:30.466406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:30.466526 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:30.466649 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:30.468063 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.469279 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.471199 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.471866 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.472006 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.477930 (+ 5924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:30.477953 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:30.477975 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:30.478028 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:30.481777 (+ 3749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:30.482785 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:30.482792 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:30.482795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:30.482959 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:30.483450 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:30.483454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:30.483456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:30.483918 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:30.484162 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:30.484541 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 19:47:30.484727 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 19:47:30.486047 9883 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:30.601892 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 9758
I20260806 19:47:30.622649 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 9894
Aug 06 19:47:30 dist-test-slave-77v9 krb5kdc[9516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.67@KRBTEST.COM
I20260806 19:47:30.643734 10042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:30.623633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36708 (local address 127.21.202.67:36503)
0806 19:47:30.623888 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:30.623898 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:30.623921 (+ 23us) server_negotiation.cc:407] Connection header received
0806 19:47:30.624108 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:30.624113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:30.624180 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:30.624326 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:30.626286 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.627097 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.629343 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:30.629869 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:30.630071 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:30.635565 (+ 5494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:30.635581 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:30.635602 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:30.635660 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:30.640159 (+ 4499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:30.641228 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:30.641238 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:30.641242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:30.641404 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:30.641824 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:30.641829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:30.641833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:30.642403 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:30.642727 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:30.643090 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 19:47:30.643357 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 19:47:30.644608 10019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:30.778353 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 9894
I20260806 19:47:30.806213 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 9529
I20260806 19:47:30.807878 9591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:30.923066 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 9529
2026-08-06T19:47:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4858 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T19:47:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:30Z Disabled control of system clock
I20260806 19:47:31.022897 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42841
--webserver_interface=127.21.202.126
--webserver_port=0
--builtin_ntp_servers=127.21.202.84:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:42841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 19:47:31.389451 10080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:31.389781 10080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:31.389874 10080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:31.397477 10080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:31.397626 10080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:31.397670 10080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:31.397725 10080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:31.397764 10080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:31.407491 10080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:42841
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:31.408885 10080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:31.410699 10080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:31.423957 10085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:31.423988 10086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:31.423914 10088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:31.424626 10080 server_base.cc:1034] running on GCE node
I20260806 19:47:31.425611 10080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:31.427330 10080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:31.430744 10080 hybrid_clock.cc:648] HybridClock initialized: now 1786045651430634 us; error 51 us; skew 500 ppm
I20260806 19:47:31.431543 10080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:47:31.435397 10080 webserver.cc:460] Webserver started at http://127.21.202.126:35723/ using document root <none> and password file <none>
I20260806 19:47:31.436672 10080 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:31.436857 10080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:31.437330 10080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:31.442186 10080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "fcec9f654178497b859a4aadeced4379"
format_stamp: "Formatted at 2026-08-06 19:47:31 on dist-test-slave-77v9"
server_key: "fb69b5a4ee60a43193504fc9ea4cb385"
server_key_iv: "a53b6b3480e37affecc7f344dab1e92d"
server_key_version: "encryptionkey@0"
I20260806 19:47:31.443121 10080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "fcec9f654178497b859a4aadeced4379"
format_stamp: "Formatted at 2026-08-06 19:47:31 on dist-test-slave-77v9"
server_key: "fb69b5a4ee60a43193504fc9ea4cb385"
server_key_iv: "a53b6b3480e37affecc7f344dab1e92d"
server_key_version: "encryptionkey@0"
I20260806 19:47:31.449188 10080 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 19:47:31.453049 10094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:31.454636 10080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:47:31.454847 10080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "fcec9f654178497b859a4aadeced4379"
format_stamp: "Formatted at 2026-08-06 19:47:31 on dist-test-slave-77v9"
server_key: "fb69b5a4ee60a43193504fc9ea4cb385"
server_key_iv: "a53b6b3480e37affecc7f344dab1e92d"
server_key_version: "encryptionkey@0"
I20260806 19:47:31.455086 10080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:31.490080 10080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:31.491153 10080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:31.491539 10080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:31.514164 10080 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:42841
I20260806 19:47:31.514204 10145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:42841 every 8 connection(s)
I20260806 19:47:31.515820 10080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:31.518168 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 10080
I20260806 19:47:31.518572 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:31.519070 22313 external_mini_cluster.cc:1428] Setting key d1439f8ec44a8e1bb97a65e3c06699af
I20260806 19:47:31.520558 10146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:31.533241 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:31.521078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48050 (local address 127.21.202.126:42841)
0806 19:47:31.522170 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:31.522182 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:31.522223 (+ 41us) server_negotiation.cc:407] Connection header received
0806 19:47:31.523835 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:31.523856 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:31.524338 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:31.524791 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:31.525426 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:31.526785 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:31.528194 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:31.528995 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:31.529204 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:31.530621 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:31.530629 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:31.530643 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:31.531240 (+ 597us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:31.531331 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:31.531355 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:31.531516 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:31.531755 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 19:47:31.532102 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":401,"threads_started":1}
I20260806 19:47:31.535099 10146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379: Bootstrap starting.
I20260806 19:47:31.538949 10146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:31.540120 10146 log.cc:824] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:31.543313 10146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379: No bootstrap required, opened a new log
I20260806 19:47:31.552124 10146 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcec9f654178497b859a4aadeced4379" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42841 } }
I20260806 19:47:31.552731 10146 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:31.552958 10146 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcec9f654178497b859a4aadeced4379, State: Initialized, Role: FOLLOWER
I20260806 19:47:31.553699 10146 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcec9f654178497b859a4aadeced4379" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42841 } }
I20260806 19:47:31.553938 10146 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:31.554042 10146 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:31.554191 10146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:31.557667 10146 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcec9f654178497b859a4aadeced4379" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42841 } }
I20260806 19:47:31.558184 10146 leader_election.cc:302] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcec9f654178497b859a4aadeced4379; no voters:
I20260806 19:47:31.558972 10146 leader_election.cc:288] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:31.559084 10151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:31.560487 10151 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [term 1 LEADER]: Becoming Leader. State: Replica: fcec9f654178497b859a4aadeced4379, State: Running, Role: LEADER
I20260806 19:47:31.561086 10151 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcec9f654178497b859a4aadeced4379" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42841 } }
I20260806 19:47:31.561547 10146 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:31.563728 10153 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcec9f654178497b859a4aadeced4379. Latest consensus state: current_term: 1 leader_uuid: "fcec9f654178497b859a4aadeced4379" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcec9f654178497b859a4aadeced4379" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42841 } } }
I20260806 19:47:31.564157 10153 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:31.564852 10152 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcec9f654178497b859a4aadeced4379" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcec9f654178497b859a4aadeced4379" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42841 } } }
I20260806 19:47:31.565141 10152 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:31.566972 10158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:31.573858 10158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:31.583575 10158 catalog_manager.cc:1347] Generated new cluster ID: 7d19ca281c0c4d4695ea078fc2dd3279
I20260806 19:47:31.583705 10158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:31.594754 10158 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:31.596858 10158 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:31.612351 10158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379: Generated new TSK 0
I20260806 19:47:31.613533 10158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:31.631211 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.65:0
--local_ip_for_outbound_sockets=127.21.202.65
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42841
--builtin_ntp_servers=127.21.202.84:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 19:47:31.996767 10171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:31.997170 10171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:31.997259 10171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:32.004678 10171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:32.004792 10171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:32.005028 10171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.65
I20260806 19:47:32.014647 10171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.202.65
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.65
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:32.015997 10171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:32.018016 10171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:32.031548 10176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:32.032374 10177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:32.032577 10179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:32.033547 10171 server_base.cc:1034] running on GCE node
I20260806 19:47:32.034262 10171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:32.035311 10171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:32.036542 10171 hybrid_clock.cc:648] HybridClock initialized: now 1786045652036487 us; error 51 us; skew 500 ppm
I20260806 19:47:32.036947 10171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:47:32.039495 10171 webserver.cc:460] Webserver started at http://127.21.202.65:34831/ using document root <none> and password file <none>
I20260806 19:47:32.040251 10171 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:32.040355 10171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:32.040681 10171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:32.043672 10171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/data/instance:
uuid: "ca48043232454320b29a6c9c6a5fbd18"
format_stamp: "Formatted at 2026-08-06 19:47:32 on dist-test-slave-77v9"
server_key: "1f94bb8db7f0a755fba8f986512abbc0"
server_key_iv: "85e5311ee200ba29470746ef33efbb3a"
server_key_version: "encryptionkey@0"
I20260806 19:47:32.044515 10171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance:
uuid: "ca48043232454320b29a6c9c6a5fbd18"
format_stamp: "Formatted at 2026-08-06 19:47:32 on dist-test-slave-77v9"
server_key: "1f94bb8db7f0a755fba8f986512abbc0"
server_key_iv: "85e5311ee200ba29470746ef33efbb3a"
server_key_version: "encryptionkey@0"
I20260806 19:47:32.050024 10171 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:47:32.053791 10185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:32.055212 10171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:47:32.055397 10171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/wal
uuid: "ca48043232454320b29a6c9c6a5fbd18"
format_stamp: "Formatted at 2026-08-06 19:47:32 on dist-test-slave-77v9"
server_key: "1f94bb8db7f0a755fba8f986512abbc0"
server_key_iv: "85e5311ee200ba29470746ef33efbb3a"
server_key_version: "encryptionkey@0"
I20260806 19:47:32.055612 10171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:32.080404 10171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:32.081427 10171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:32.081768 10171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:32.082902 10171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:32.084532 10171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:32.084625 10171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:32.084761 10171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:32.084865 10171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:32.122205 10171 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.65:36153
I20260806 19:47:32.122361 10297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.65:36153 every 8 connection(s)
I20260806 19:47:32.123617 10171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/data/info.pb
I20260806 19:47:32.126227 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 10171
I20260806 19:47:32.126475 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/wal/instance
I20260806 19:47:32.127174 22313 external_mini_cluster.cc:1428] Setting key 35be91a79dda8d7fd182d3ac7b0091ea
I20260806 19:47:32.131219 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.66:0
--local_ip_for_outbound_sockets=127.21.202.66
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42841
--builtin_ntp_servers=127.21.202.84:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 19:47:32.138166 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:32.126767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.65:37865 (local address 127.21.202.126:42841)
0806 19:47:32.127329 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:32.127336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:32.128602 (+ 1266us) server_negotiation.cc:407] Connection header received
0806 19:47:32.131178 (+ 2576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:32.131185 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:32.131263 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:32.131416 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:32.132621 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:32.133431 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:32.135036 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:32.135443 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:32.135580 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:32.137254 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:32.137260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:32.137264 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:32.137284 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:32.137324 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:32.137337 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:32.137439 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:32.137686 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 19:47:32.137804 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":269,"threads_started":1}
I20260806 19:47:32.139272 10302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:32.127243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42841 (local address 127.21.202.65:37865)
0806 19:47:32.128407 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:32.128462 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:32.130825 (+ 2363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:32.131688 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:32.131698 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:32.131803 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:32.132441 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:32.132460 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:32.133609 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:32.133613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:32.134862 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:32.134875 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:32.135690 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:32.135698 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:32.135938 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:32.136778 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:47:32.136781 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:32.136896 (+ 115us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:47:32.137000 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:32.137453 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:32.137466 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:32.137481 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 19:47:32.137677 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 19:47:32.138116 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":318,"threads_started":1}
I20260806 19:47:32.141084 10299 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42841
I20260806 19:47:32.141438 10299 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:32.142290 10299 heartbeater.cc:511] Master 127.21.202.126:42841 requested a full tablet report, sending...
I20260806 19:47:32.145133 10111 ts_manager.cc:194] Registered new tserver with Master: ca48043232454320b29a6c9c6a5fbd18 (127.21.202.65:36153)
I20260806 19:47:32.147415 10111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.202.65:37865
W20260806 19:47:32.484201 10303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:32.484591 10303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:32.484686 10303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:32.493180 10303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:32.493301 10303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:32.493419 10303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.66
I20260806 19:47:32.503255 10303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.202.66
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.66
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:32.504602 10303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:32.506570 10303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:32.521415 10311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:32.521394 10309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:32.522235 10303 server_base.cc:1034] running on GCE node
W20260806 19:47:32.522488 10308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:32.523319 10303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:32.524436 10303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:32.525748 10303 hybrid_clock.cc:648] HybridClock initialized: now 1786045652525650 us; error 49 us; skew 500 ppm
I20260806 19:47:32.526341 10303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:47:32.529199 10303 webserver.cc:460] Webserver started at http://127.21.202.66:37903/ using document root <none> and password file <none>
I20260806 19:47:32.530021 10303 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:32.530169 10303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:32.530529 10303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:32.533783 10303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/data/instance:
uuid: "257d19c8be3b433e999430b0c78d0e23"
format_stamp: "Formatted at 2026-08-06 19:47:32 on dist-test-slave-77v9"
server_key: "7b1c7cb6c7620beb2a3a177f3a7addcb"
server_key_iv: "b1a27964285750f7dfaf44f7a94d6e75"
server_key_version: "encryptionkey@0"
I20260806 19:47:32.534662 10303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance:
uuid: "257d19c8be3b433e999430b0c78d0e23"
format_stamp: "Formatted at 2026-08-06 19:47:32 on dist-test-slave-77v9"
server_key: "7b1c7cb6c7620beb2a3a177f3a7addcb"
server_key_iv: "b1a27964285750f7dfaf44f7a94d6e75"
server_key_version: "encryptionkey@0"
I20260806 19:47:32.540546 10303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 19:47:32.544502 10317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:32.545969 10303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:47:32.546172 10303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/wal
uuid: "257d19c8be3b433e999430b0c78d0e23"
format_stamp: "Formatted at 2026-08-06 19:47:32 on dist-test-slave-77v9"
server_key: "7b1c7cb6c7620beb2a3a177f3a7addcb"
server_key_iv: "b1a27964285750f7dfaf44f7a94d6e75"
server_key_version: "encryptionkey@0"
I20260806 19:47:32.546408 10303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:32.561225 10303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:32.562232 10303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:32.562557 10303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:32.563671 10303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:32.565346 10303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:32.565469 10303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:32.565613 10303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:32.565703 10303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:32.605691 10303 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.66:41685
I20260806 19:47:32.605804 10429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.66:41685 every 8 connection(s)
I20260806 19:47:32.607090 10303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/data/info.pb
I20260806 19:47:32.617293 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 10303
I20260806 19:47:32.617493 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/wal/instance
I20260806 19:47:32.618268 22313 external_mini_cluster.cc:1428] Setting key 5136569ced4821c100103d551050f7e1
I20260806 19:47:32.621140 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:32.610128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.66:39825 (local address 127.21.202.126:42841)
0806 19:47:32.610330 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:32.610336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:32.611818 (+ 1482us) server_negotiation.cc:407] Connection header received
0806 19:47:32.614089 (+ 2271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:32.614095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:32.614202 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:32.614338 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:32.615589 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:32.616351 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:32.618129 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:32.618611 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:32.618746 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:32.620290 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:32.620295 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:32.620299 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:32.620318 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:32.620351 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:32.620364 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:32.620461 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:32.620690 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 19:47:32.620791 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 19:47:32.622080 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.202.67:0
--local_ip_for_outbound_sockets=127.21.202.67
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42841
--builtin_ntp_servers=127.21.202.84:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 19:47:32.622196 10432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:32.610628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42841 (local address 127.21.202.66:39825)
0806 19:47:32.611631 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:32.611684 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:32.613777 (+ 2093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:32.614575 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:32.614584 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:32.614685 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:32.615402 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:32.615420 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:32.616512 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:32.616524 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:32.617984 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:32.617996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:32.618864 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:32.618874 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:32.619090 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:32.619936 (+ 846us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:47:32.619939 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:32.619986 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:47:32.620067 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:32.620459 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:32.620471 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:32.620481 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 19:47:32.620685 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 19:47:32.621126 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":330,"threads_started":1}
I20260806 19:47:32.624251 10430 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42841
I20260806 19:47:32.624606 10430 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:32.625531 10430 heartbeater.cc:511] Master 127.21.202.126:42841 requested a full tablet report, sending...
I20260806 19:47:32.627249 10111 ts_manager.cc:194] Registered new tserver with Master: 257d19c8be3b433e999430b0c78d0e23 (127.21.202.66:41685)
I20260806 19:47:32.628580 10111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.202.66:39825
W20260806 19:47:32.995918 10433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:32.996295 10433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:32.996462 10433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:33.004305 10433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:33.004491 10433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:33.004683 10433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.202.67
I20260806 19:47:33.014878 10433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.202.67
--webserver_port=0
--tserver_master_addrs=127.21.202.126:42841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.202.67
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:33.016309 10433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:33.018205 10433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:33.032341 10440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:33.032352 10438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:33.032491 10442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:33.033509 10433 server_base.cc:1034] running on GCE node
I20260806 19:47:33.034222 10433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:33.035539 10433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:33.036895 10433 hybrid_clock.cc:648] HybridClock initialized: now 1786045653036762 us; error 54 us; skew 500 ppm
I20260806 19:47:33.037240 10433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:47:33.040343 10433 webserver.cc:460] Webserver started at http://127.21.202.67:36941/ using document root <none> and password file <none>
I20260806 19:47:33.041188 10433 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:33.041326 10433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:33.041663 10433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:33.044649 10433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/data/instance:
uuid: "48608f15e35645d3bc9567549ce64189"
format_stamp: "Formatted at 2026-08-06 19:47:33 on dist-test-slave-77v9"
server_key: "697c1ff39c713259704e735ce0199f36"
server_key_iv: "883b60c6089419ba7741d966ccc71722"
server_key_version: "encryptionkey@0"
I20260806 19:47:33.045583 10433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance:
uuid: "48608f15e35645d3bc9567549ce64189"
format_stamp: "Formatted at 2026-08-06 19:47:33 on dist-test-slave-77v9"
server_key: "697c1ff39c713259704e735ce0199f36"
server_key_iv: "883b60c6089419ba7741d966ccc71722"
server_key_version: "encryptionkey@0"
I20260806 19:47:33.051295 10433 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:47:33.055194 10448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:33.056697 10433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:47:33.056999 10433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/wal
uuid: "48608f15e35645d3bc9567549ce64189"
format_stamp: "Formatted at 2026-08-06 19:47:33 on dist-test-slave-77v9"
server_key: "697c1ff39c713259704e735ce0199f36"
server_key_iv: "883b60c6089419ba7741d966ccc71722"
server_key_version: "encryptionkey@0"
I20260806 19:47:33.057233 10433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:33.084983 10433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:33.085983 10433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:33.086356 10433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:33.087625 10433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:47:33.089440 10433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:47:33.089608 10433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:33.089763 10433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:47:33.089874 10433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:33.130110 10433 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.67:43981
I20260806 19:47:33.130172 10560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.67:43981 every 8 connection(s)
I20260806 19:47:33.131667 10433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/data/info.pb
I20260806 19:47:33.139839 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 10433
I20260806 19:47:33.140071 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/wal/instance
I20260806 19:47:33.140666 22313 external_mini_cluster.cc:1428] Setting key 435635d9b65b18735a645976ca33b51c
I20260806 19:47:33.147328 10563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.134687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:33729 (local address 127.21.202.126:42841)
0806 19:47:33.135195 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:33.135200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:33.136437 (+ 1237us) server_negotiation.cc:407] Connection header received
0806 19:47:33.138580 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:33.138587 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:33.138694 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:33.138842 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:33.140380 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.141391 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.143910 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.144622 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.144763 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.146388 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:33.146394 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:33.146398 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:33.146418 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:33.146475 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:33.146490 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:33.146582 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:33.146880 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 19:47:33.146982 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":265,"threads_started":1}
I20260806 19:47:33.148393 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.135258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:42841 (local address 127.21.202.67:33729)
0806 19:47:33.136245 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:33.136291 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:33.138265 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:33.139096 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:33.139105 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:33.139211 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:47:33.140147 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:33.140177 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.141600 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.141608 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:33.143725 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:33.143738 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.144957 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.144965 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:33.145204 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.145989 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:47:33.145992 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:47:33.146035 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:47:33.146131 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:47:33.146624 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:47:33.146633 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:47:33.146640 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 19:47:33.146816 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 19:47:33.147165 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":309,"threads_started":1}
I20260806 19:47:33.150240 10561 heartbeater.cc:347] Connected to a master server at 127.21.202.126:42841
I20260806 19:47:33.150609 10561 heartbeater.cc:464] Registering TS with master...
I20260806 19:47:33.151188 10299 heartbeater.cc:503] Master 127.21.202.126:42841 was elected leader, sending a full tablet report...
I20260806 19:47:33.151719 10561 heartbeater.cc:511] Master 127.21.202.126:42841 requested a full tablet report, sending...
I20260806 19:47:33.153522 10111 ts_manager.cc:194] Registered new tserver with Master: 48608f15e35645d3bc9567549ce64189 (127.21.202.67:43981)
I20260806 19:47:33.154289 10111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.202.67:33729
I20260806 19:47:33.157963 22313 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:47:33.169639 10563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.162868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48058 (local address 127.21.202.126:42841)
0806 19:47:33.163092 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:33.163100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:33.163622 (+ 522us) server_negotiation.cc:407] Connection header received
0806 19:47:33.163754 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:33.163758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:33.163861 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:33.163967 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:33.164477 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.165312 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.166798 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.167366 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.167502 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.168861 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:33.168868 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:33.168873 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:33.168904 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:33.168937 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:33.168950 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:33.169055 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:33.169204 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 19:47:33.169306 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 19:47:33.175118 10111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:47:33.182941 10111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:47:33.216991 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.197215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58430 (local address 127.21.202.66:41685)
0806 19:47:33.205773 (+ 8558us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:33.205781 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:33.206264 (+ 483us) server_negotiation.cc:407] Connection header received
0806 19:47:33.206830 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:33.206842 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:33.207183 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:33.207423 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:33.208603 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.210498 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.213912 (+ 3414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.215398 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.215565 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.215833 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:33.216371 (+ 538us) server_negotiation.cc:299] Negotiation successful
0806 19:47:33.216551 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8401,"thread_start_us":8241,"threads_started":1}
I20260806 19:47:33.217046 10577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.196655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:41685 (local address 127.0.0.1:58430)
0806 19:47:33.206082 (+ 9427us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:33.206121 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:33.206427 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:33.207764 (+ 1337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:33.207777 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:33.207809 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:33.208368 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:33.208383 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.210739 (+ 2356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.210747 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:33.213530 (+ 2783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:33.213569 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.215965 (+ 2396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.215970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:33.216125 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.216158 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 19:47:33.216325 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 19:47:33.216549 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9108,"mutex_wait_us":48,"thread_start_us":10908,"threads_started":1}
I20260806 19:47:33.217585 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.198566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.21.202.67:43981)
0806 19:47:33.205282 (+ 6716us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:33.205295 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:33.208128 (+ 2833us) server_negotiation.cc:407] Connection header received
0806 19:47:33.208407 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:33.208422 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:33.208688 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:33.208986 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:33.209581 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.211333 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.214344 (+ 3011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.215764 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.215897 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.216123 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:33.217005 (+ 882us) server_negotiation.cc:299] Negotiation successful
0806 19:47:33.217188 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6453,"thread_start_us":6247,"threads_started":1}
I20260806 19:47:33.217573 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.197057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55104 (local address 127.21.202.65:36153)
0806 19:47:33.197723 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:33.197748 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:33.206658 (+ 8910us) server_negotiation.cc:407] Connection header received
0806 19:47:33.207037 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:33.207047 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:33.207342 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:33.207548 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:33.208702 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.210115 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.213808 (+ 3693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.215644 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.215821 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.216119 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:33.217005 (+ 886us) server_negotiation.cc:299] Negotiation successful
0806 19:47:33.217185 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":393,"threads_started":1}
I20260806 19:47:33.218420 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.203964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.67:43981 (local address 127.0.0.1:53500)
0806 19:47:33.207964 (+ 4000us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:33.207985 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:33.208202 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:33.208962 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:33.208969 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:33.208994 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:33.209359 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:33.209372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.211508 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.211516 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:33.214077 (+ 2561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:33.214093 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.216350 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.216358 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:33.216641 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.216792 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 19:47:33.217835 (+ 1043us) client_negotiation.cc:240] Negotiation successful
0806 19:47:33.218032 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3850,"mutex_wait_us":156,"thread_start_us":2071,"threads_started":1}
I20260806 19:47:33.218577 10579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.196659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:36153 (local address 127.0.0.1:55104)
0806 19:47:33.206509 (+ 9850us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:33.206535 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:33.206790 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:33.207935 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:33.207940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:33.207967 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:33.208465 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:33.208478 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.210297 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.210313 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:33.213468 (+ 3155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:33.213485 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.216364 (+ 2879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.216369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:33.216640 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.216792 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 19:47:33.218086 (+ 1294us) client_negotiation.cc:240] Negotiation successful
0806 19:47:33.218209 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9641,"mutex_wait_us":704,"thread_start_us":6598,"threads_started":1}
I20260806 19:47:33.222148 10496 tablet_service.cc:1467] Processing CreateTablet for tablet f11cc93bf41e4935800c051590c68242 (DEFAULT_TABLE table=test-table [id=5a134ac2632f406f8199427c34b3e152]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:33.221479 10365 tablet_service.cc:1467] Processing CreateTablet for tablet f11cc93bf41e4935800c051590c68242 (DEFAULT_TABLE table=test-table [id=5a134ac2632f406f8199427c34b3e152]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:33.223529 10496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f11cc93bf41e4935800c051590c68242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:33.223531 10365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f11cc93bf41e4935800c051590c68242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:33.224020 10233 tablet_service.cc:1467] Processing CreateTablet for tablet f11cc93bf41e4935800c051590c68242 (DEFAULT_TABLE table=test-table [id=5a134ac2632f406f8199427c34b3e152]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:47:33.225414 10233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f11cc93bf41e4935800c051590c68242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:33.240128 10581 tablet_bootstrap.cc:492] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23: Bootstrap starting.
I20260806 19:47:33.245004 10582 tablet_bootstrap.cc:492] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189: Bootstrap starting.
I20260806 19:47:33.245950 10580 tablet_bootstrap.cc:492] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18: Bootstrap starting.
I20260806 19:47:33.246713 10581 tablet_bootstrap.cc:654] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:33.248108 10581 log.cc:824] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:33.249822 10582 tablet_bootstrap.cc:654] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:33.250062 10580 tablet_bootstrap.cc:654] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:33.251137 10582 log.cc:824] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:33.251176 10580 log.cc:824] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:33.252048 10581 tablet_bootstrap.cc:492] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23: No bootstrap required, opened a new log
I20260806 19:47:33.252420 10581 ts_tablet_manager.cc:1397] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260806 19:47:33.260567 10580 tablet_bootstrap.cc:492] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18: No bootstrap required, opened a new log
I20260806 19:47:33.260567 10582 tablet_bootstrap.cc:492] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189: No bootstrap required, opened a new log
I20260806 19:47:33.260922 10582 ts_tablet_manager.cc:1397] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.003s
I20260806 19:47:33.260948 10580 ts_tablet_manager.cc:1397] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.002s
I20260806 19:47:33.264775 10581 raft_consensus.cc:348] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257d19c8be3b433e999430b0c78d0e23" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 41685 } } peers { permanent_uuid: "ca48043232454320b29a6c9c6a5fbd18" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36153 } } peers { permanent_uuid: "48608f15e35645d3bc9567549ce64189" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43981 } }
I20260806 19:47:33.265355 10581 raft_consensus.cc:374] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:33.265472 10581 raft_consensus.cc:729] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257d19c8be3b433e999430b0c78d0e23, State: Initialized, Role: FOLLOWER
I20260806 19:47:33.266170 10581 consensus_queue.cc:260] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257d19c8be3b433e999430b0c78d0e23" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 41685 } } peers { permanent_uuid: "ca48043232454320b29a6c9c6a5fbd18" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36153 } } peers { permanent_uuid: "48608f15e35645d3bc9567549ce64189" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43981 } }
I20260806 19:47:33.267542 10430 heartbeater.cc:503] Master 127.21.202.126:42841 was elected leader, sending a full tablet report...
I20260806 19:47:33.268390 10581 ts_tablet_manager.cc:1428] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23: Time spent starting tablet: real 0.016s user 0.008s sys 0.008s
I20260806 19:47:33.272480 10582 raft_consensus.cc:348] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257d19c8be3b433e999430b0c78d0e23" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 41685 } } peers { permanent_uuid: "ca48043232454320b29a6c9c6a5fbd18" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36153 } } peers { permanent_uuid: "48608f15e35645d3bc9567549ce64189" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43981 } }
I20260806 19:47:33.272467 10580 raft_consensus.cc:348] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257d19c8be3b433e999430b0c78d0e23" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 41685 } } peers { permanent_uuid: "ca48043232454320b29a6c9c6a5fbd18" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36153 } } peers { permanent_uuid: "48608f15e35645d3bc9567549ce64189" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43981 } }
I20260806 19:47:33.272986 10582 raft_consensus.cc:374] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:33.272986 10580 raft_consensus.cc:374] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:33.273101 10580 raft_consensus.cc:729] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca48043232454320b29a6c9c6a5fbd18, State: Initialized, Role: FOLLOWER
I20260806 19:47:33.273098 10582 raft_consensus.cc:729] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48608f15e35645d3bc9567549ce64189, State: Initialized, Role: FOLLOWER
I20260806 19:47:33.273769 10582 consensus_queue.cc:260] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257d19c8be3b433e999430b0c78d0e23" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 41685 } } peers { permanent_uuid: "ca48043232454320b29a6c9c6a5fbd18" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36153 } } peers { permanent_uuid: "48608f15e35645d3bc9567549ce64189" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43981 } }
I20260806 19:47:33.273766 10580 consensus_queue.cc:260] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257d19c8be3b433e999430b0c78d0e23" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 41685 } } peers { permanent_uuid: "ca48043232454320b29a6c9c6a5fbd18" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36153 } } peers { permanent_uuid: "48608f15e35645d3bc9567549ce64189" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43981 } }
I20260806 19:47:33.281751 10561 heartbeater.cc:503] Master 127.21.202.126:42841 was elected leader, sending a full tablet report...
I20260806 19:47:33.282541 10582 ts_tablet_manager.cc:1428] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189: Time spent starting tablet: real 0.021s user 0.016s sys 0.004s
I20260806 19:47:33.282537 10580 ts_tablet_manager.cc:1428] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18: Time spent starting tablet: real 0.021s user 0.018s sys 0.002s
W20260806 19:47:33.359377 10431 tablet.cc:2369] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:47:33.376447 10300 tablet.cc:2369] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:47:33.383816 10562 tablet.cc:2369] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:47:33.581024 10588 raft_consensus.cc:482] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:47:33.581399 10588 raft_consensus.cc:504] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257d19c8be3b433e999430b0c78d0e23" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 41685 } } peers { permanent_uuid: "ca48043232454320b29a6c9c6a5fbd18" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36153 } } peers { permanent_uuid: "48608f15e35645d3bc9567549ce64189" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43981 } }
I20260806 19:47:33.583273 10588 leader_election.cc:288] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 257d19c8be3b433e999430b0c78d0e23 (127.21.202.66:41685), ca48043232454320b29a6c9c6a5fbd18 (127.21.202.65:36153)
I20260806 19:47:33.589184 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.583794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:49131 (local address 127.21.202.66:41685)
0806 19:47:33.584013 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:33.584020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:33.584040 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:47:33.584230 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:33.584242 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:33.584331 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:33.584448 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:33.584920 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.585625 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.587294 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.588194 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.588387 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.588462 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:33.588751 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 19:47:33.588902 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 19:47:33.589188 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.583555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.66:41685 (local address 127.21.202.67:49131)
0806 19:47:33.583902 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:33.583924 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:33.584051 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:33.584473 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:33.584482 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:33.584522 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:33.584733 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:33.584741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.585763 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.585771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:33.587120 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:33.587132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.588379 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.588385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:33.588587 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.588612 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 19:47:33.588722 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 19:47:33.588882 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181}
I20260806 19:47:33.590152 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.583969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.202.67:56453 (local address 127.21.202.65:36153)
0806 19:47:33.584182 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:33.584190 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:33.584612 (+ 422us) server_negotiation.cc:407] Connection header received
0806 19:47:33.584869 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:33.584877 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:33.584963 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:33.585072 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:47:33.585654 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.586383 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.588225 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.589152 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.589353 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.589428 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:33.589741 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 19:47:33.589895 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 19:47:33.590188 10589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.583749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.65:36153 (local address 127.21.202.67:56453)
0806 19:47:33.584494 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0806 19:47:33.584509 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 19:47:33.584683 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:47:33.585105 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:47:33.585116 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:47:33.585145 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:47:33.585481 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:33.585490 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.586570 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.586576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:33.588019 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:47:33.588031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.589348 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.589355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:47:33.589556 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.589582 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 19:47:33.589699 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 19:47:33.589854 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":432,"threads_started":1}
I20260806 19:47:33.590794 10385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f11cc93bf41e4935800c051590c68242" candidate_uuid: "48608f15e35645d3bc9567549ce64189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257d19c8be3b433e999430b0c78d0e23" is_pre_election: true
I20260806 19:47:33.591411 10385 raft_consensus.cc:2454] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48608f15e35645d3bc9567549ce64189 in term 0.
I20260806 19:47:33.591542 10253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f11cc93bf41e4935800c051590c68242" candidate_uuid: "48608f15e35645d3bc9567549ce64189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca48043232454320b29a6c9c6a5fbd18" is_pre_election: true
I20260806 19:47:33.592101 10253 raft_consensus.cc:2454] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48608f15e35645d3bc9567549ce64189 in term 0.
I20260806 19:47:33.592180 10449 leader_election.cc:302] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257d19c8be3b433e999430b0c78d0e23, 48608f15e35645d3bc9567549ce64189; no voters:
I20260806 19:47:33.592689 10588 raft_consensus.cc:2781] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:47:33.593047 10588 raft_consensus.cc:482] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:47:33.593192 10588 raft_consensus.cc:3037] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:33.595441 10588 raft_consensus.cc:504] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257d19c8be3b433e999430b0c78d0e23" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 41685 } } peers { permanent_uuid: "ca48043232454320b29a6c9c6a5fbd18" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36153 } } peers { permanent_uuid: "48608f15e35645d3bc9567549ce64189" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43981 } }
I20260806 19:47:33.596068 10588 leader_election.cc:288] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [CANDIDATE]: Term 1 election: Requested vote from peers 257d19c8be3b433e999430b0c78d0e23 (127.21.202.66:41685), ca48043232454320b29a6c9c6a5fbd18 (127.21.202.65:36153)
I20260806 19:47:33.596688 10253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f11cc93bf41e4935800c051590c68242" candidate_uuid: "48608f15e35645d3bc9567549ce64189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca48043232454320b29a6c9c6a5fbd18"
I20260806 19:47:33.596688 10385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f11cc93bf41e4935800c051590c68242" candidate_uuid: "48608f15e35645d3bc9567549ce64189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257d19c8be3b433e999430b0c78d0e23"
I20260806 19:47:33.596961 10385 raft_consensus.cc:3037] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:33.596951 10253 raft_consensus.cc:3037] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:33.600582 10253 raft_consensus.cc:2454] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48608f15e35645d3bc9567549ce64189 in term 1.
I20260806 19:47:33.600582 10385 raft_consensus.cc:2454] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48608f15e35645d3bc9567549ce64189 in term 1.
I20260806 19:47:33.601140 10449 leader_election.cc:302] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257d19c8be3b433e999430b0c78d0e23, 48608f15e35645d3bc9567549ce64189; no voters:
I20260806 19:47:33.601473 10588 raft_consensus.cc:2781] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:33.601970 10588 raft_consensus.cc:686] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [term 1 LEADER]: Becoming Leader. State: Replica: 48608f15e35645d3bc9567549ce64189, State: Running, Role: LEADER
I20260806 19:47:33.602479 10588 consensus_queue.cc:237] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257d19c8be3b433e999430b0c78d0e23" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 41685 } } peers { permanent_uuid: "ca48043232454320b29a6c9c6a5fbd18" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36153 } } peers { permanent_uuid: "48608f15e35645d3bc9567549ce64189" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43981 } }
I20260806 19:47:33.607625 10111 catalog_manager.cc:5495] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48608f15e35645d3bc9567549ce64189 (127.21.202.67). New cstate: current_term: 1 leader_uuid: "48608f15e35645d3bc9567549ce64189" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257d19c8be3b433e999430b0c78d0e23" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 41685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca48043232454320b29a6c9c6a5fbd18" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48608f15e35645d3bc9567549ce64189" member_type: VOTER last_known_addr { host: "127.21.202.67" port: 43981 } health_report { overall_health: HEALTHY } } }
I20260806 19:47:33.679219 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.672772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53506 (local address 127.21.202.67:43981)
0806 19:47:33.673084 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:33.673091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:33.673382 (+ 291us) server_negotiation.cc:407] Connection header received
0806 19:47:33.673569 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:33.673573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:33.673670 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:33.673819 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:47:33.674358 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.675203 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.676734 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.677274 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.677391 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.677935 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:47:33.678516 (+ 581us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:47:33.678637 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:33.678762 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 19:47:33.678841 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260806 19:47:33.687790 10253 raft_consensus.cc:1261] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18 [term 1 FOLLOWER]: Refusing update from remote peer 48608f15e35645d3bc9567549ce64189: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:47:33.687790 10385 raft_consensus.cc:1261] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23 [term 1 FOLLOWER]: Refusing update from remote peer 48608f15e35645d3bc9567549ce64189: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:47:33.688871 10590 consensus_queue.cc:1035] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [LEADER]: Connected to new peer: Peer: permanent_uuid: "257d19c8be3b433e999430b0c78d0e23" member_type: VOTER last_known_addr { host: "127.21.202.66" port: 41685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:47:33.689276 10588 consensus_queue.cc:1035] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca48043232454320b29a6c9c6a5fbd18" member_type: VOTER last_known_addr { host: "127.21.202.65" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:47:33.702451 10595 mvcc.cc:205] Tried to move back new op lower bound from 7315642997495287808 to 7315642997159829504. Current Snapshot: MvccSnapshot[applied={T|T < 7315642997495287808}]
I20260806 19:47:33.702451 10596 mvcc.cc:205] Tried to move back new op lower bound from 7315642997495287808 to 7315642997159829504. Current Snapshot: MvccSnapshot[applied={T|T < 7315642997495287808}]
I20260806 19:47:33.705192 10594 mvcc.cc:205] Tried to move back new op lower bound from 7315642997495287808 to 7315642997159829504. Current Snapshot: MvccSnapshot[applied={T|T < 7315642997495287808 or (T in {7315642997495287808})}]
I20260806 19:47:33.716224 10111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:47:33.716535 10111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:47:33.720171 10111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fcec9f654178497b859a4aadeced4379: Sending DeleteTablet for 3 replicas of tablet f11cc93bf41e4935800c051590c68242
I20260806 19:47:33.721441 10365 tablet_service.cc:1514] Processing DeleteTablet for tablet f11cc93bf41e4935800c051590c68242 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:33 UTC) from {username='slave'} at 127.0.0.1:58430
I20260806 19:47:33.721503 10233 tablet_service.cc:1514] Processing DeleteTablet for tablet f11cc93bf41e4935800c051590c68242 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:33 UTC) from {username='slave'} at 127.0.0.1:55104
I20260806 19:47:33.721710 10496 tablet_service.cc:1514] Processing DeleteTablet for tablet f11cc93bf41e4935800c051590c68242 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:47:33 UTC) from {username='slave'} at 127.0.0.1:53500
I20260806 19:47:33.722489 10605 tablet_replica.cc:330] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23: stopping tablet replica
I20260806 19:47:33.723048 10606 tablet_replica.cc:330] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189: stopping tablet replica
I20260806 19:47:33.723068 10604 tablet_replica.cc:330] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18: stopping tablet replica
I20260806 19:47:33.723510 10604 raft_consensus.cc:2229] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:47:33.723518 10606 raft_consensus.cc:2229] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:47:33.723641 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 10171
I20260806 19:47:33.724210 10604 raft_consensus.cc:2258] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:33.724298 10606 raft_consensus.cc:2258] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:33.725003 10605 raft_consensus.cc:2229] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:47:33.725613 10605 raft_consensus.cc:2258] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:47:33.727437 10604 ts_tablet_manager.cc:1905] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:33.728582 10606 ts_tablet_manager.cc:1905] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:33.729382 10605 ts_tablet_manager.cc:1905] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:47:33.735167 10604 ts_tablet_manager.cc:1918] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:33.735435 10604 log.cc:1196] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-0/wal/wals/f11cc93bf41e4935800c051590c68242
I20260806 19:47:33.736116 10604 ts_tablet_manager.cc:1939] T f11cc93bf41e4935800c051590c68242 P ca48043232454320b29a6c9c6a5fbd18: Deleting consensus metadata
I20260806 19:47:33.737169 10605 ts_tablet_manager.cc:1918] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:33.737191 10606 ts_tablet_manager.cc:1918] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:47:33.737370 10605 log.cc:1196] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-1/wal/wals/f11cc93bf41e4935800c051590c68242
I20260806 19:47:33.737376 10606 log.cc:1196] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189: Deleting WAL directory at /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786045310306758-22313-0/minicluster-data/ts-2/wal/wals/f11cc93bf41e4935800c051590c68242
I20260806 19:47:33.737973 10605 ts_tablet_manager.cc:1939] T f11cc93bf41e4935800c051590c68242 P 257d19c8be3b433e999430b0c78d0e23: Deleting consensus metadata
I20260806 19:47:33.737996 10606 ts_tablet_manager.cc:1939] T f11cc93bf41e4935800c051590c68242 P 48608f15e35645d3bc9567549ce64189: Deleting consensus metadata
I20260806 19:47:33.738590 10098 catalog_manager.cc:4841] TS ca48043232454320b29a6c9c6a5fbd18 (127.21.202.65:36153): tablet f11cc93bf41e4935800c051590c68242 (table test-table [id=5a134ac2632f406f8199427c34b3e152]) successfully deleted
I20260806 19:47:33.739815 10097 catalog_manager.cc:4841] TS 48608f15e35645d3bc9567549ce64189 (127.21.202.67:43981): tablet f11cc93bf41e4935800c051590c68242 (table test-table [id=5a134ac2632f406f8199427c34b3e152]) successfully deleted
I20260806 19:47:33.741366 10095 catalog_manager.cc:4841] TS 257d19c8be3b433e999430b0c78d0e23 (127.21.202.66:41685): tablet f11cc93bf41e4935800c051590c68242 (table test-table [id=5a134ac2632f406f8199427c34b3e152]) successfully deleted
I20260806 19:47:33.745699 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.725133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55110 (local address 127.21.202.65:36153)
0806 19:47:33.725330 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:33.725338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:33.726261 (+ 923us) server_negotiation.cc:407] Connection header received
0806 19:47:33.726611 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:33.726617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:33.726741 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:33.726826 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:33.738613 (+ 11787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.739703 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.741863 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.742507 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.742651 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.743847 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:33.743860 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:33.743878 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:33.744694 (+ 816us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:33.744767 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:33.744793 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:33.744935 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:33.745203 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 19:47:33.745326 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 19:47:33.746480 10293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:33.841167 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 10171
I20260806 19:47:33.861896 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 10303
I20260806 19:47:33.870364 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.862749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58444 (local address 127.21.202.66:41685)
0806 19:47:33.862942 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:33.862949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:33.863107 (+ 158us) server_negotiation.cc:407] Connection header received
0806 19:47:33.863297 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:33.863304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:33.863438 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:33.863553 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:33.864015 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.865173 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.866475 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.867212 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.867370 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:33.868631 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:33.868643 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:33.868662 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:33.869528 (+ 866us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:33.869589 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:33.869620 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:33.869737 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:33.869878 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 19:47:33.870004 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 19:47:33.871091 10425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:33.967036 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 10303
I20260806 19:47:33.993273 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 10433
I20260806 19:47:34.002331 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:33.994284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53516 (local address 127.21.202.67:43981)
0806 19:47:33.994532 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:33.994537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:33.994556 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:47:33.994676 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:33.994679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:33.994803 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:33.994917 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:33.995457 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.996361 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.998558 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:33.999137 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:33.999245 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:34.000573 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:34.000585 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:34.000600 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:34.001357 (+ 757us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:34.001406 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:34.001426 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:34.001516 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:34.001743 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 19:47:34.001867 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 19:47:34.003217 10556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:34.128087 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 10433
I20260806 19:47:34.147802 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 10080
I20260806 19:47:34.149106 10141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:34.259711 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 10080
2026-08-06T19:47:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3332 ms)
[----------] 6 tests from EncryptionPolicyTest (24008 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:34 dist-test-slave-77v9 krb5kdc[10614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:34 dist-test-slave-77v9 krb5kdc[10614](info): set up 2 sockets
Aug 06 19:47:34 dist-test-slave-77v9 krb5kdc[10614](info): commencing operation
krb5kdc: starting...
W20260806 19:47:35.448460 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:35 dist-test-slave-77v9 krb5kdc[10614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:47:35.791493 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:36.145951 10627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:36.146296 10627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:36.146522 10627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:36.154273 10627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:36.154417 10627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:36.154524 10627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:36.154634 10627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:36.154738 10627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:36.165364 10627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36889
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:36.166831 10627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:36.168296 10627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:36.181381 10635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:36.181363 10633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:36.181943 10632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:36.182344 10627 server_base.cc:1034] running on GCE node
I20260806 19:47:36.183167 10627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:36.184623 10627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:36.185899 10627 hybrid_clock.cc:648] HybridClock initialized: now 1786045656185879 us; error 78 us; skew 500 ppm
I20260806 19:47:36.186297 10627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:36 dist-test-slave-77v9 krb5kdc[10614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045656, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:36.194418 10627 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:47:36.196132 10627 webserver.cc:460] Webserver started at http://127.0.0.1:46473/ using document root <none> and password file <none>
I20260806 19:47:36.196964 10627 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:36.197122 10627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:36.197460 10627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:36.200871 10627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "ce3e0ea4e30d40f2bc1ce39d4eb64a58"
format_stamp: "Formatted at 2026-08-06 19:47:36 on dist-test-slave-77v9"
server_key: "185f286797eea33829f07810980176ba"
server_key_iv: "ddfd0d919bac739a615955a3cccc3d57"
server_key_version: "encryptionkey@0"
I20260806 19:47:36.201843 10627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "ce3e0ea4e30d40f2bc1ce39d4eb64a58"
format_stamp: "Formatted at 2026-08-06 19:47:36 on dist-test-slave-77v9"
server_key: "185f286797eea33829f07810980176ba"
server_key_iv: "ddfd0d919bac739a615955a3cccc3d57"
server_key_version: "encryptionkey@0"
I20260806 19:47:36.207887 10627 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:47:36.211861 10642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:36.213413 10627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:47:36.213634 10627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "ce3e0ea4e30d40f2bc1ce39d4eb64a58"
format_stamp: "Formatted at 2026-08-06 19:47:36 on dist-test-slave-77v9"
server_key: "185f286797eea33829f07810980176ba"
server_key_iv: "ddfd0d919bac739a615955a3cccc3d57"
server_key_version: "encryptionkey@0"
I20260806 19:47:36.213869 10627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:36.238821 10627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:36.244104 10627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:36.244443 10627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:36.265089 10627 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36889
I20260806 19:47:36.265128 10693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36889 every 8 connection(s)
I20260806 19:47:36.266538 10627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:36.270501 10694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:36.276377 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 10627
I20260806 19:47:36.276615 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:36.277503 22313 external_mini_cluster.cc:1428] Setting key 3275024dbdc4891203da523ab22b5c90
I20260806 19:47:36.282630 10694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58: Bootstrap starting.
I20260806 19:47:36.286592 10694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:36.287817 10694 log.cc:824] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:36.291347 10694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58: No bootstrap required, opened a new log
Aug 06 19:47:36 dist-test-slave-77v9 krb5kdc[10614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 19:47:36.299876 10694 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3e0ea4e30d40f2bc1ce39d4eb64a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } }
I20260806 19:47:36.300266 10694 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:36.300369 10694 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce3e0ea4e30d40f2bc1ce39d4eb64a58, State: Initialized, Role: FOLLOWER
I20260806 19:47:36.301158 10694 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3e0ea4e30d40f2bc1ce39d4eb64a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } }
I20260806 19:47:36.301486 10694 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:36.301640 10694 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:36.301855 10694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:36.303948 10697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:36.279812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53231 (local address 127.0.0.1:36889)
0806 19:47:36.280730 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:36.280742 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:36.280806 (+ 64us) server_negotiation.cc:407] Connection header received
0806 19:47:36.282957 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:36.282982 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:36.283409 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:36.283762 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:36.285878 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:36.287313 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:36.289151 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:36.289913 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:36.290053 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:36.295193 (+ 5140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:36.295201 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:36.295217 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:36.295265 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:36.298751 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:36.299657 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:36.299664 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:36.299671 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:36.299836 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:36.300445 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:36.300453 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:36.300457 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:36.301204 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:36.301561 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:36.302083 (+ 522us) server_negotiation.cc:299] Negotiation successful
0806 19:47:36.302459 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":433,"threads_started":1}
I20260806 19:47:36.305142 10694 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3e0ea4e30d40f2bc1ce39d4eb64a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } }
I20260806 19:47:36.305648 10694 leader_election.cc:302] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce3e0ea4e30d40f2bc1ce39d4eb64a58; no voters:
I20260806 19:47:36.306360 10694 leader_election.cc:288] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:36.306519 10699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:36.308007 10699 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [term 1 LEADER]: Becoming Leader. State: Replica: ce3e0ea4e30d40f2bc1ce39d4eb64a58, State: Running, Role: LEADER
I20260806 19:47:36.308529 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:47:36.308668 10699 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3e0ea4e30d40f2bc1ce39d4eb64a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } }
I20260806 19:47:36.308934 10694 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:36.315837 10700 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce3e0ea4e30d40f2bc1ce39d4eb64a58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3e0ea4e30d40f2bc1ce39d4eb64a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } } }
I20260806 19:47:36.315994 10701 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce3e0ea4e30d40f2bc1ce39d4eb64a58. Latest consensus state: current_term: 1 leader_uuid: "ce3e0ea4e30d40f2bc1ce39d4eb64a58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3e0ea4e30d40f2bc1ce39d4eb64a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } } }
I20260806 19:47:36.316636 10700 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:36.317410 10701 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:36.317631 10712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:36.324260 10712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:36.338151 10712 catalog_manager.cc:1347] Generated new cluster ID: c791c2c0a1954a2eab6eb368654e9167
I20260806 19:47:36.338418 10712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:36.356272 10712 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:36.358870 10712 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:36.367319 10697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:36.337487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34429 (local address 127.0.0.1:36889)
0806 19:47:36.337671 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:36.337677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:36.348182 (+ 10505us) server_negotiation.cc:407] Connection header received
0806 19:47:36.348731 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:36.348740 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:36.348910 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:36.348992 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:36.351215 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:36.352436 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:36.354411 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:36.355217 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:36.355360 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:36.359262 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:36.359275 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:36.359281 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:36.359327 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:36.363633 (+ 4306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:36.364702 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:36.364708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:36.364712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:36.364925 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:36.365479 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:36.365486 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:36.365489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:36.365883 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:36.366192 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:36.366601 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 19:47:36.366818 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 19:47:36.380235 10712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce3e0ea4e30d40f2bc1ce39d4eb64a58: Generated new TSK 0
I20260806 19:47:36.381436 10712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:36.392227 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 10627
I20260806 19:47:36.394345 10689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:36.494544 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 10627
W20260806 19:47:36.514110 10612 connection.cc:535] client connection to 127.0.0.1:36889 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36889): Connection reset by peer (error 104)
2026-08-06T19:47:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2234 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:36 dist-test-slave-77v9 krb5kdc[10726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:36 dist-test-slave-77v9 krb5kdc[10726](info): set up 2 sockets
Aug 06 19:47:36 dist-test-slave-77v9 krb5kdc[10726](info): commencing operation
krb5kdc: starting...
W20260806 19:47:37.673671 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:37 dist-test-slave-77v9 krb5kdc[10726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:47:38.022038 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43131
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:38.378772 10739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:38.379098 10739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:38.379191 10739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:38.386703 10739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:38.386828 10739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:38.386876 10739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:38.386955 10739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:38.387003 10739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:38.397292 10739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43131
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:38.398627 10739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:38.399927 10739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:38.415567 10745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:38.415673 10747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:38.416013 10744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:38.416354 10739 server_base.cc:1034] running on GCE node
I20260806 19:47:38.417428 10739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:38.419245 10739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:38.420507 10739 hybrid_clock.cc:648] HybridClock initialized: now 1786045658420452 us; error 56 us; skew 500 ppm
I20260806 19:47:38.421094 10739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:38 dist-test-slave-77v9 krb5kdc[10726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045658, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:38.425397 10739 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:47:38.427570 10739 webserver.cc:460] Webserver started at http://127.0.0.1:45181/ using document root <none> and password file <none>
I20260806 19:47:38.428615 10739 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:38.428771 10739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:38.429149 10739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:38.433053 10739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "49c2b87d15c347b09105c00840036765"
format_stamp: "Formatted at 2026-08-06 19:47:38 on dist-test-slave-77v9"
server_key: "a334f46328360cc05e8a23aa5c5edc63"
server_key_iv: "1ae4bdc1c00c70e9910bcd4ac0797598"
server_key_version: "encryptionkey@0"
I20260806 19:47:38.434237 10739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "49c2b87d15c347b09105c00840036765"
format_stamp: "Formatted at 2026-08-06 19:47:38 on dist-test-slave-77v9"
server_key: "a334f46328360cc05e8a23aa5c5edc63"
server_key_iv: "1ae4bdc1c00c70e9910bcd4ac0797598"
server_key_version: "encryptionkey@0"
I20260806 19:47:38.442606 10739 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.005s
I20260806 19:47:38.447929 10754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:38.449568 10739 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 19:47:38.449774 10739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "49c2b87d15c347b09105c00840036765"
format_stamp: "Formatted at 2026-08-06 19:47:38 on dist-test-slave-77v9"
server_key: "a334f46328360cc05e8a23aa5c5edc63"
server_key_iv: "1ae4bdc1c00c70e9910bcd4ac0797598"
server_key_version: "encryptionkey@0"
I20260806 19:47:38.449998 10739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:38.482671 10739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:38.487367 10739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:38.487675 10739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:38.507870 10739 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43131
I20260806 19:47:38.507977 10805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43131 every 8 connection(s)
I20260806 19:47:38.509413 10739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:38.513464 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:38.516896 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 10739
I20260806 19:47:38.517125 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:38.517753 22313 external_mini_cluster.cc:1428] Setting key 891ede49021c26ea74a009807674f649
I20260806 19:47:38.528266 10806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765: Bootstrap starting.
I20260806 19:47:38.532564 10806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:38.533962 10806 log.cc:824] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:38.534608 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:38.519800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49565 (local address 127.0.0.1:43131)
0806 19:47:38.520643 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:38.520654 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:38.520696 (+ 42us) server_negotiation.cc:407] Connection header received
0806 19:47:38.522431 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:38.522455 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:38.522851 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:38.523163 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:38.523857 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:38.526148 (+ 2291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:38.528309 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:38.529526 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:38.529970 (+ 444us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:38.531775 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:38.531786 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:38.531804 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:38.532396 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:38.532499 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:38.532527 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:38.532658 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:38.533086 (+ 428us) server_negotiation.cc:299] Negotiation successful
0806 19:47:38.533326 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":405,"threads_started":1}
I20260806 19:47:38.537752 10806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765: No bootstrap required, opened a new log
I20260806 19:47:38.539397 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:47:38.549491 10806 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c2b87d15c347b09105c00840036765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43131 } }
I20260806 19:47:38.549916 10806 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:38.550071 10806 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c2b87d15c347b09105c00840036765, State: Initialized, Role: FOLLOWER
I20260806 19:47:38.550793 10806 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c2b87d15c347b09105c00840036765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43131 } }
I20260806 19:47:38.551107 10806 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:38.551273 10806 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:38.551491 10806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:38.553774 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:38.546044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55337 (local address 127.0.0.1:43131)
0806 19:47:38.546269 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:38.546277 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:38.546716 (+ 439us) server_negotiation.cc:407] Connection header received
0806 19:47:38.546884 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:38.546890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:38.546986 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:38.547109 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:38.547598 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:38.548782 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:38.550426 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:38.551121 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:38.551275 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:38.552539 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:38.552545 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:38.552550 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:38.552572 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:38.552642 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:38.552664 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:38.552794 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:38.553358 (+ 564us) server_negotiation.cc:299] Negotiation successful
0806 19:47:38.553419 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 19:47:38.554535 10806 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c2b87d15c347b09105c00840036765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43131 } }
I20260806 19:47:38.555150 10806 leader_election.cc:302] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49c2b87d15c347b09105c00840036765; no voters:
I20260806 19:47:38.556178 10806 leader_election.cc:288] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:38.556296 10816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:38.557055 10816 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [term 1 LEADER]: Becoming Leader. State: Replica: 49c2b87d15c347b09105c00840036765, State: Running, Role: LEADER
I20260806 19:47:38.557704 10816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c2b87d15c347b09105c00840036765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43131 } }
I20260806 19:47:38.558708 10806 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:38.560698 10817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49c2b87d15c347b09105c00840036765" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c2b87d15c347b09105c00840036765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43131 } } }
I20260806 19:47:38.561870 10817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:38.561537 10818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49c2b87d15c347b09105c00840036765. Latest consensus state: current_term: 1 leader_uuid: "49c2b87d15c347b09105c00840036765" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c2b87d15c347b09105c00840036765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43131 } } }
I20260806 19:47:38.562343 10818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:38.573511 10821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:38.579411 10821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:38.587865 10821 catalog_manager.cc:1347] Generated new cluster ID: 12440ceb832541c4b606adc91ac3627d
I20260806 19:47:38.588002 10821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:38.621491 10821 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:38.623067 10821 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:38.631322 10821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49c2b87d15c347b09105c00840036765: Generated new TSK 0
I20260806 19:47:38.632195 10821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:38.642758 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 10739
W20260806 19:47:38.644690 10801 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49565
W20260806 19:47:38.645602 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 10739: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:47:38.645807 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 10739
W20260806 19:47:38.665040 10724 connection.cc:535] client connection to 127.0.0.1:43131 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43131): Connection reset by peer (error 104)
2026-08-06T19:47:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2146 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:38 dist-test-slave-77v9 krb5kdc[10838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:38 dist-test-slave-77v9 krb5kdc[10838](info): set up 2 sockets
Aug 06 19:47:38 dist-test-slave-77v9 krb5kdc[10838](info): commencing operation
krb5kdc: starting...
W20260806 19:47:39.843022 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.001s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:40 dist-test-slave-77v9 krb5kdc[10838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:47:40.189462 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38241
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:40.566217 10851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:40.566569 10851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:40.566797 10851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:40.574918 10851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:40.575070 10851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:40.575183 10851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:40.575307 10851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:40.575385 10851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:40.586674 10851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38241
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:40.588124 10851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:40.589517 10851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:40.602773 10856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:40.603019 10857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:40.603010 10859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:40.603777 10851 server_base.cc:1034] running on GCE node
I20260806 19:47:40.604614 10851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:40.606601 10851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:40.607862 10851 hybrid_clock.cc:648] HybridClock initialized: now 1786045660607797 us; error 58 us; skew 500 ppm
I20260806 19:47:40.608289 10851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:40 dist-test-slave-77v9 krb5kdc[10838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045660, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:40.611855 10851 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:47:40.613760 10851 webserver.cc:460] Webserver started at http://127.0.0.1:43595/ using document root <none> and password file <none>
I20260806 19:47:40.614512 10851 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:40.614648 10851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:40.615005 10851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:40.618476 10851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "43aae8d709c64f08a669bb745b19e82a"
format_stamp: "Formatted at 2026-08-06 19:47:40 on dist-test-slave-77v9"
server_key: "43e85bc855549c0af74312c551a59a67"
server_key_iv: "5ec82981a0f77b5e18ee6c14ba3e89bb"
server_key_version: "encryptionkey@0"
I20260806 19:47:40.619331 10851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "43aae8d709c64f08a669bb745b19e82a"
format_stamp: "Formatted at 2026-08-06 19:47:40 on dist-test-slave-77v9"
server_key: "43e85bc855549c0af74312c551a59a67"
server_key_iv: "5ec82981a0f77b5e18ee6c14ba3e89bb"
server_key_version: "encryptionkey@0"
I20260806 19:47:40.625136 10851 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 19:47:40.628927 10866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:40.630370 10851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:47:40.630584 10851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "43aae8d709c64f08a669bb745b19e82a"
format_stamp: "Formatted at 2026-08-06 19:47:40 on dist-test-slave-77v9"
server_key: "43e85bc855549c0af74312c551a59a67"
server_key_iv: "5ec82981a0f77b5e18ee6c14ba3e89bb"
server_key_version: "encryptionkey@0"
I20260806 19:47:40.630802 10851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:40.637848 10851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:40.642561 10851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:40.642901 10851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:40.663247 10851 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38241
I20260806 19:47:40.663293 10917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38241 every 8 connection(s)
I20260806 19:47:40.664728 10851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:40.668618 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:40.674645 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 10851
I20260806 19:47:40.674877 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:40.675310 22313 external_mini_cluster.cc:1428] Setting key 69c271e27f7eb620dd6938ef7b8fb04d
I20260806 19:47:40.681413 10918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a: Bootstrap starting.
I20260806 19:47:40.684993 10921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:40.677376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57491 (local address 127.0.0.1:38241)
0806 19:47:40.678368 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:40.678390 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:40.678443 (+ 53us) server_negotiation.cc:407] Connection header received
0806 19:47:40.680887 (+ 2444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:40.680917 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:40.681382 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:40.681872 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:40.682935 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:40.682946 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:40.682963 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:40.682983 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:40.683059 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:40.683095 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:40.683349 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:40.683500 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 19:47:40.683711 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":427,"threads_started":1}
I20260806 19:47:40.685784 10918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:40.687013 10918 log.cc:824] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:40.689316 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:47:40.690788 10918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a: No bootstrap required, opened a new log
I20260806 19:47:40.697412 10921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:40.694474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52919 (local address 127.0.0.1:38241)
0806 19:47:40.694670 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:40.694677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:40.695076 (+ 399us) server_negotiation.cc:407] Connection header received
0806 19:47:40.695221 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:40.695232 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:40.695296 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:40.695447 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:40.696436 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:40.696443 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:40.696448 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:40.696469 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:40.696526 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:40.696547 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:40.696687 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:40.696980 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 19:47:40.697093 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 19:47:40.700498 10918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43aae8d709c64f08a669bb745b19e82a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38241 } }
I20260806 19:47:40.701088 10918 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:40.701232 10918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43aae8d709c64f08a669bb745b19e82a, State: Initialized, Role: FOLLOWER
I20260806 19:47:40.701989 10918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43aae8d709c64f08a669bb745b19e82a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38241 } }
I20260806 19:47:40.702289 10918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:40.702483 10918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:40.702692 10918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:40.705328 10918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43aae8d709c64f08a669bb745b19e82a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38241 } }
I20260806 19:47:40.705994 10918 leader_election.cc:302] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43aae8d709c64f08a669bb745b19e82a; no voters:
I20260806 19:47:40.706689 10918 leader_election.cc:288] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:40.706856 10928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:40.708320 10928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [term 1 LEADER]: Becoming Leader. State: Replica: 43aae8d709c64f08a669bb745b19e82a, State: Running, Role: LEADER
I20260806 19:47:40.708938 10928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43aae8d709c64f08a669bb745b19e82a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38241 } }
I20260806 19:47:40.709347 10918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:40.712316 10930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43aae8d709c64f08a669bb745b19e82a. Latest consensus state: current_term: 1 leader_uuid: "43aae8d709c64f08a669bb745b19e82a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43aae8d709c64f08a669bb745b19e82a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38241 } } }
I20260806 19:47:40.712762 10930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:40.713376 10929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43aae8d709c64f08a669bb745b19e82a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43aae8d709c64f08a669bb745b19e82a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38241 } } }
I20260806 19:47:40.713676 10929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:40.715307 10936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:40.721084 10936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:40.730495 10936 catalog_manager.cc:1347] Generated new cluster ID: abfa55b429dd46fcbc61eb862b043003
I20260806 19:47:40.730643 10936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:40.751036 10936 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:40.751569 10936 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:40.761377 10936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43aae8d709c64f08a669bb745b19e82a: Generated new TSK 0
I20260806 19:47:40.762290 10936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:40.777508 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 10851
W20260806 19:47:40.778990 10913 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57491
W20260806 19:47:40.779999 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 10851: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:47:40.780355 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 10851
2026-08-06T19:47:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2135 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:40 dist-test-slave-77v9 krb5kdc[10949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:40 dist-test-slave-77v9 krb5kdc[10949](info): set up 2 sockets
Aug 06 19:47:40 dist-test-slave-77v9 krb5kdc[10949](info): commencing operation
krb5kdc: starting...
W20260806 19:47:41.959229 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.002s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:42 dist-test-slave-77v9 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:47:42.297246 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45291
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:42.649618 10962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:42.649968 10962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:42.650077 10962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:42.657836 10962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:42.657972 10962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:42.658023 10962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:42.658087 10962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:42.658130 10962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:42.668288 10962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:42.669682 10962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:42.671008 10962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:42.683382 10967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:42.683365 10970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:42.683385 10968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:42.684242 10962 server_base.cc:1034] running on GCE node
I20260806 19:47:42.685076 10962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:42.686550 10962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:42.687762 10962 hybrid_clock.cc:648] HybridClock initialized: now 1786045662687726 us; error 26 us; skew 500 ppm
I20260806 19:47:42.688154 10962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:42 dist-test-slave-77v9 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045662, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:42.694370 10962 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:47:42.696096 10962 webserver.cc:460] Webserver started at http://127.0.0.1:41745/ using document root <none> and password file <none>
I20260806 19:47:42.696863 10962 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:42.697029 10962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:42.697359 10962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:42.700537 10962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "9bd3223d59294d37b3ff45df41126ad3"
format_stamp: "Formatted at 2026-08-06 19:47:42 on dist-test-slave-77v9"
server_key: "1fc96cd34eff24a5199cab8648165945"
server_key_iv: "5b6c71ff8ea77137833dbe42016179e9"
server_key_version: "encryptionkey@0"
I20260806 19:47:42.701442 10962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "9bd3223d59294d37b3ff45df41126ad3"
format_stamp: "Formatted at 2026-08-06 19:47:42 on dist-test-slave-77v9"
server_key: "1fc96cd34eff24a5199cab8648165945"
server_key_iv: "5b6c71ff8ea77137833dbe42016179e9"
server_key_version: "encryptionkey@0"
I20260806 19:47:42.706873 10962 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:47:42.710844 10977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:42.712306 10962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:47:42.712507 10962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "9bd3223d59294d37b3ff45df41126ad3"
format_stamp: "Formatted at 2026-08-06 19:47:42 on dist-test-slave-77v9"
server_key: "1fc96cd34eff24a5199cab8648165945"
server_key_iv: "5b6c71ff8ea77137833dbe42016179e9"
server_key_version: "encryptionkey@0"
I20260806 19:47:42.712724 10962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:42.727339 10962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:42.732137 10962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:42.732685 10962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:42.752599 10962 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45291
I20260806 19:47:42.752658 11028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45291 every 8 connection(s)
I20260806 19:47:42.754192 10962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:42.758841 11029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:42.761387 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 10962
I20260806 19:47:42.761593 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:42.762220 22313 external_mini_cluster.cc:1428] Setting key 35e346f964d50e8f33b681ac623c736f
I20260806 19:47:42.774350 11029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3: Bootstrap starting.
Aug 06 19:47:42 dist-test-slave-77v9 krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 19:47:42.778761 11029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:42.780010 11029 log.cc:824] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:42.783592 11029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3: No bootstrap required, opened a new log
I20260806 19:47:42.787899 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:42.764152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52423 (local address 127.0.0.1:45291)
0806 19:47:42.765007 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:42.765018 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:42.765058 (+ 40us) server_negotiation.cc:407] Connection header received
0806 19:47:42.766842 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:42.766865 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:42.767274 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:42.767601 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:42.769527 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:42.770893 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:42.772791 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:42.773577 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:42.773729 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:42.779081 (+ 5352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:42.779090 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:42.779106 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:42.779155 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:42.782955 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:42.784045 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:42.784066 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:42.784092 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:42.784276 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:42.784762 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:42.784768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:42.784776 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:42.785455 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:42.785823 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:42.786331 (+ 508us) server_negotiation.cc:299] Negotiation successful
0806 19:47:42.786805 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":401,"threads_started":1}
I20260806 19:47:42.792038 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:47:42.792447 11029 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd3223d59294d37b3ff45df41126ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45291 } }
I20260806 19:47:42.792878 11029 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:42.793000 11029 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bd3223d59294d37b3ff45df41126ad3, State: Initialized, Role: FOLLOWER
I20260806 19:47:42.793576 11029 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd3223d59294d37b3ff45df41126ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45291 } }
I20260806 19:47:42.793835 11029 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:42.793984 11029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:42.794163 11029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:42.796710 11029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd3223d59294d37b3ff45df41126ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45291 } }
I20260806 19:47:42.797461 11029 leader_election.cc:302] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bd3223d59294d37b3ff45df41126ad3; no voters:
I20260806 19:47:42.798163 11029 leader_election.cc:288] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:42.798309 11039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:42.800710 11029 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:42.804683 11039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [term 1 LEADER]: Becoming Leader. State: Replica: 9bd3223d59294d37b3ff45df41126ad3, State: Running, Role: LEADER
I20260806 19:47:42.805536 11039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd3223d59294d37b3ff45df41126ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45291 } }
I20260806 19:47:42.814244 11041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bd3223d59294d37b3ff45df41126ad3. Latest consensus state: current_term: 1 leader_uuid: "9bd3223d59294d37b3ff45df41126ad3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd3223d59294d37b3ff45df41126ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45291 } } }
I20260806 19:47:42.814718 11041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:42.814219 11040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bd3223d59294d37b3ff45df41126ad3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd3223d59294d37b3ff45df41126ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45291 } } }
I20260806 19:47:42.815140 11040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3 [sys.catalog]: This master's current role is: LEADER
W20260806 19:47:42.819303 11053 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:47:42.819465 11053 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:47:42.819624 11054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:42.820353 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:42.797168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54865 (local address 127.0.0.1:45291)
0806 19:47:42.797420 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:42.797430 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:42.797864 (+ 434us) server_negotiation.cc:407] Connection header received
0806 19:47:42.798144 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:42.798154 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:42.798262 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:42.798424 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:42.800355 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:42.801969 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:42.804546 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:42.805280 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:42.805435 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:42.809742 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:42.809755 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:42.809764 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:42.809827 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:42.815757 (+ 5930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:42.817003 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:42.817008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:42.817011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:42.817169 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:42.817745 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:42.817751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:42.817754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:42.818269 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:42.819790 (+ 1521us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:42.819909 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 19:47:42.820077 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 19:47:42.822628 11054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:42.832525 11054 catalog_manager.cc:1347] Generated new cluster ID: cd0386f9e82c4d639589c7c25e9c00bc
I20260806 19:47:42.832681 11054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:42.854184 11054 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:42.856122 11054 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:42.865765 11054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bd3223d59294d37b3ff45df41126ad3: Generated new TSK 0
I20260806 19:47:42.867092 11054 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:42.881779 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 10962
I20260806 19:47:42.883093 11024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:42.982372 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 10962
2026-08-06T19:47:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2201 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:43 dist-test-slave-77v9 krb5kdc[11061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:43 dist-test-slave-77v9 krb5kdc[11061](info): set up 2 sockets
Aug 06 19:47:43 dist-test-slave-77v9 krb5kdc[11061](info): commencing operation
krb5kdc: starting...
W20260806 19:47:44.146431 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:44 dist-test-slave-77v9 krb5kdc[11061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:47:44.489275 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39567
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:44.851855 11074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:44.852176 11074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:44.852370 11074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:44.860335 11074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:44.860478 11074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:44.860584 11074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:44.860683 11074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:44.860775 11074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:44.871410 11074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:44.872900 11074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:44.874264 11074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:44.887766 11079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:44.887794 11080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:44.888468 11082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:44.888545 11074 server_base.cc:1034] running on GCE node
I20260806 19:47:44.889498 11074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:44.890977 11074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:44.892213 11074 hybrid_clock.cc:648] HybridClock initialized: now 1786045664892177 us; error 35 us; skew 500 ppm
I20260806 19:47:44.892575 11074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:44 dist-test-slave-77v9 krb5kdc[11061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045664, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:44.895968 11074 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:47:44.897567 11074 webserver.cc:460] Webserver started at http://127.0.0.1:44929/ using document root <none> and password file <none>
I20260806 19:47:44.898248 11074 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:44.898362 11074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:44.898655 11074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:44.902262 11074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "eeaed55a6a624deeb94fbf4783a28598"
format_stamp: "Formatted at 2026-08-06 19:47:44 on dist-test-slave-77v9"
server_key: "c30ef44b4ea950421161f9e8dc976af0"
server_key_iv: "c2abf7532c708d103317faa91e003298"
server_key_version: "encryptionkey@0"
I20260806 19:47:44.903111 11074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "eeaed55a6a624deeb94fbf4783a28598"
format_stamp: "Formatted at 2026-08-06 19:47:44 on dist-test-slave-77v9"
server_key: "c30ef44b4ea950421161f9e8dc976af0"
server_key_iv: "c2abf7532c708d103317faa91e003298"
server_key_version: "encryptionkey@0"
I20260806 19:47:44.908752 11074 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 19:47:44.912569 11089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:44.914431 11074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:47:44.914654 11074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "eeaed55a6a624deeb94fbf4783a28598"
format_stamp: "Formatted at 2026-08-06 19:47:44 on dist-test-slave-77v9"
server_key: "c30ef44b4ea950421161f9e8dc976af0"
server_key_iv: "c2abf7532c708d103317faa91e003298"
server_key_version: "encryptionkey@0"
I20260806 19:47:44.914909 11074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:44.938898 11074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:44.943707 11074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:44.944027 11074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:44.964366 11074 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39567
I20260806 19:47:44.964395 11140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39567 every 8 connection(s)
I20260806 19:47:44.965842 11074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:44.970505 11141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:44.974793 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 11074
I20260806 19:47:44.975013 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:44.975625 22313 external_mini_cluster.cc:1428] Setting key e924de6164837a683b4bd3c2f6bd40da
I20260806 19:47:44.986153 11141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598: Bootstrap starting.
I20260806 19:47:44.990134 11141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:44.991364 11141 log.cc:824] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:44.991703 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:44.977973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36823 (local address 127.0.0.1:39567)
0806 19:47:44.978874 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:44.978888 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:44.978932 (+ 44us) server_negotiation.cc:407] Connection header received
0806 19:47:44.980713 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:44.980737 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:44.981364 (+ 627us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:44.981820 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:44.982426 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:44.984504 (+ 2078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:44.986666 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:44.987377 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:44.987588 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:44.989009 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:44.989018 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:44.989032 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:44.989597 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:44.989680 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:44.989707 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:44.989854 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:44.990195 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 19:47:44.990516 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":423,"threads_started":1}
I20260806 19:47:44.995023 11141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598: No bootstrap required, opened a new log
I20260806 19:47:44.995862 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:47:45.004508 11141 raft_consensus.cc:348] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeaed55a6a624deeb94fbf4783a28598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39567 } }
I20260806 19:47:45.005093 11141 raft_consensus.cc:374] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:45.005227 11141 raft_consensus.cc:729] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeaed55a6a624deeb94fbf4783a28598, State: Initialized, Role: FOLLOWER
I20260806 19:47:45.006047 11141 consensus_queue.cc:260] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeaed55a6a624deeb94fbf4783a28598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39567 } }
I20260806 19:47:45.006381 11141 raft_consensus.cc:388] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:45.006564 11141 raft_consensus.cc:482] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:45.006748 11141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:45.010201 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:45.002138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35381 (local address 127.0.0.1:39567)
0806 19:47:45.002660 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:45.002671 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:45.003184 (+ 513us) server_negotiation.cc:407] Connection header received
0806 19:47:45.003396 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:45.003400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:45.003470 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:45.003560 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:45.004282 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:45.005168 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:45.007301 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:45.007847 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:45.007968 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:45.009374 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:45.009379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:45.009383 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:45.009403 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:45.009444 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:45.009461 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:45.009539 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:45.009731 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 19:47:45.009823 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260806 19:47:45.010279 11141 raft_consensus.cc:504] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeaed55a6a624deeb94fbf4783a28598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39567 } }
I20260806 19:47:45.010998 11141 leader_election.cc:302] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eeaed55a6a624deeb94fbf4783a28598; no voters:
I20260806 19:47:45.011914 11141 leader_election.cc:288] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:45.012024 11151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:45.012746 11151 raft_consensus.cc:686] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [term 1 LEADER]: Becoming Leader. State: Replica: eeaed55a6a624deeb94fbf4783a28598, State: Running, Role: LEADER
I20260806 19:47:45.013551 11151 consensus_queue.cc:237] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeaed55a6a624deeb94fbf4783a28598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39567 } }
I20260806 19:47:45.015038 11141 sys_catalog.cc:564] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:45.017787 11152 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eeaed55a6a624deeb94fbf4783a28598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeaed55a6a624deeb94fbf4783a28598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39567 } } }
I20260806 19:47:45.018510 11153 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eeaed55a6a624deeb94fbf4783a28598. Latest consensus state: current_term: 1 leader_uuid: "eeaed55a6a624deeb94fbf4783a28598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeaed55a6a624deeb94fbf4783a28598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39567 } } }
I20260806 19:47:45.019276 11152 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:45.019346 11153 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:45.023667 11158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:45.029311 11158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:45.038980 11158 catalog_manager.cc:1347] Generated new cluster ID: 280eb61b12ea478ea2b68b692f5dfe25
I20260806 19:47:45.039121 11158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:45.065393 11158 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:45.067211 11158 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:45.079936 11158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eeaed55a6a624deeb94fbf4783a28598: Generated new TSK 0
I20260806 19:47:45.081229 11158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:45.090847 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11074
W20260806 19:47:45.092098 11136 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36823
W20260806 19:47:45.092864 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11074: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:47:45.093000 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 11074
2026-08-06T19:47:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2110 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:45 dist-test-slave-77v9 krb5kdc[11172](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:45 dist-test-slave-77v9 krb5kdc[11172](info): set up 2 sockets
Aug 06 19:47:45 dist-test-slave-77v9 krb5kdc[11172](info): commencing operation
krb5kdc: starting...
W20260806 19:47:46.327739 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:46 dist-test-slave-77v9 krb5kdc[11172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:47:46.689385 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41125
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:47.062404 11185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:47.062767 11185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:47.062873 11185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:47.070744 11185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:47.070894 11185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:47.070950 11185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:47.071017 11185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:47.071074 11185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:47.081899 11185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:47.083361 11185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:47.084718 11185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:47.098374 11193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:47.098400 11190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:47.098538 11191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:47.099826 11185 server_base.cc:1034] running on GCE node
I20260806 19:47:47.100768 11185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:47.102465 11185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:47.103720 11185 hybrid_clock.cc:648] HybridClock initialized: now 1786045667103652 us; error 55 us; skew 500 ppm
I20260806 19:47:47.104192 11185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:47 dist-test-slave-77v9 krb5kdc[11172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045667, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:47.107941 11185 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:47:47.109823 11185 webserver.cc:460] Webserver started at http://127.0.0.1:37177/ using document root <none> and password file <none>
I20260806 19:47:47.110642 11185 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:47.110816 11185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:47.111179 11185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:47.114674 11185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "b1a5bd5b2fdb494b981ac8bcc1d8d0ff"
format_stamp: "Formatted at 2026-08-06 19:47:47 on dist-test-slave-77v9"
server_key: "0b4bb80166cff71cc5e79dbfb66ed241"
server_key_iv: "8ef8e59d41c137ee79e4d25efd82fdd1"
server_key_version: "encryptionkey@0"
I20260806 19:47:47.115863 11185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "b1a5bd5b2fdb494b981ac8bcc1d8d0ff"
format_stamp: "Formatted at 2026-08-06 19:47:47 on dist-test-slave-77v9"
server_key: "0b4bb80166cff71cc5e79dbfb66ed241"
server_key_iv: "8ef8e59d41c137ee79e4d25efd82fdd1"
server_key_version: "encryptionkey@0"
I20260806 19:47:47.122002 11185 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 19:47:47.126310 11200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:47.128229 11185 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 19:47:47.128490 11185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "b1a5bd5b2fdb494b981ac8bcc1d8d0ff"
format_stamp: "Formatted at 2026-08-06 19:47:47 on dist-test-slave-77v9"
server_key: "0b4bb80166cff71cc5e79dbfb66ed241"
server_key_iv: "8ef8e59d41c137ee79e4d25efd82fdd1"
server_key_version: "encryptionkey@0"
I20260806 19:47:47.128754 11185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:47.135427 11185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:47.140627 11185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:47.141083 11185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:47.162468 11185 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41125
I20260806 19:47:47.162528 11251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41125 every 8 connection(s)
I20260806 19:47:47.164008 11185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:47.168007 11252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:47.174041 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 11185
I20260806 19:47:47.174292 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:47.174755 22313 external_mini_cluster.cc:1428] Setting key 2161922b4ce5dd36efcdb7959c44f86b
I20260806 19:47:47.179697 11252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff: Bootstrap starting.
I20260806 19:47:47.183274 11252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:47.183382 11255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:47.176756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56925 (local address 127.0.0.1:41125)
0806 19:47:47.177834 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:47.177847 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:47.177897 (+ 50us) server_negotiation.cc:407] Connection header received
0806 19:47:47.179650 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:47.179674 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:47.180108 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:47.180554 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:47.181727 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:47.181733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:47.181747 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:47.181773 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:47.181833 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:47.181855 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:47.182097 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:47.182241 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 19:47:47.182446 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":498,"threads_started":1}
I20260806 19:47:47.185071 11252 log.cc:824] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:47.188524 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:47:47.189029 11252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff: No bootstrap required, opened a new log
I20260806 19:47:47.197396 11255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:47.194491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49705 (local address 127.0.0.1:41125)
0806 19:47:47.194710 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:47.194718 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:47.195108 (+ 390us) server_negotiation.cc:407] Connection header received
0806 19:47:47.195267 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:47.195274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:47.195348 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:47.195476 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:47.196537 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:47.196545 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:47.196550 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:47.196576 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:47.196629 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:47.196649 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:47.196802 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:47.197004 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 19:47:47.197078 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:47:47.199977 11252 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a5bd5b2fdb494b981ac8bcc1d8d0ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41125 } }
I20260806 19:47:47.200363 11252 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:47.200438 11252 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a5bd5b2fdb494b981ac8bcc1d8d0ff, State: Initialized, Role: FOLLOWER
I20260806 19:47:47.201152 11252 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a5bd5b2fdb494b981ac8bcc1d8d0ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41125 } }
I20260806 19:47:47.201506 11252 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:47.201676 11252 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:47.201857 11252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:47.204149 11252 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a5bd5b2fdb494b981ac8bcc1d8d0ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41125 } }
I20260806 19:47:47.204689 11252 leader_election.cc:302] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1a5bd5b2fdb494b981ac8bcc1d8d0ff; no voters:
I20260806 19:47:47.205416 11252 leader_election.cc:288] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:47.205574 11262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:47.206435 11262 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [term 1 LEADER]: Becoming Leader. State: Replica: b1a5bd5b2fdb494b981ac8bcc1d8d0ff, State: Running, Role: LEADER
I20260806 19:47:47.207072 11262 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a5bd5b2fdb494b981ac8bcc1d8d0ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41125 } }
I20260806 19:47:47.207719 11252 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:47.210240 11263 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1a5bd5b2fdb494b981ac8bcc1d8d0ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a5bd5b2fdb494b981ac8bcc1d8d0ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41125 } } }
I20260806 19:47:47.210661 11263 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:47.210619 11264 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1a5bd5b2fdb494b981ac8bcc1d8d0ff. Latest consensus state: current_term: 1 leader_uuid: "b1a5bd5b2fdb494b981ac8bcc1d8d0ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a5bd5b2fdb494b981ac8bcc1d8d0ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41125 } } }
I20260806 19:47:47.211028 11264 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:47.214778 11271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:47.220041 11271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:47.230849 11271 catalog_manager.cc:1347] Generated new cluster ID: 67c9ffdda4cd48efb157bc216e428624
I20260806 19:47:47.231019 11271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:47.260468 11271 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:47.261065 11271 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:47.282804 11271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1a5bd5b2fdb494b981ac8bcc1d8d0ff: Generated new TSK 0
I20260806 19:47:47.284025 11271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:47.297760 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11185
W20260806 19:47:47.299109 11247 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56925
W20260806 19:47:47.299932 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11185: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:47:47.300127 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 11185
2026-08-06T19:47:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2207 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:47 dist-test-slave-77v9 krb5kdc[11283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:47 dist-test-slave-77v9 krb5kdc[11283](info): set up 2 sockets
Aug 06 19:47:47 dist-test-slave-77v9 krb5kdc[11283](info): commencing operation
krb5kdc: starting...
W20260806 19:47:48.480962 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.005s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:48 dist-test-slave-77v9 krb5kdc[11283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:47:48.825068 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41629
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:49.179651 11296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:49.180079 11296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:49.180244 11296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:49.187824 11296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:49.188002 11296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:49.188108 11296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:49.188217 11296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:49.188295 11296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:49.198567 11296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:49.200034 11296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:49.201418 11296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:49.214190 11301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:49.214150 11302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:49.214109 11304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:49.215104 11296 server_base.cc:1034] running on GCE node
I20260806 19:47:49.215874 11296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:49.217505 11296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:49.218726 11296 hybrid_clock.cc:648] HybridClock initialized: now 1786045669218642 us; error 74 us; skew 500 ppm
I20260806 19:47:49.219069 11296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:49 dist-test-slave-77v9 krb5kdc[11283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045669, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:49.222396 11296 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:47:49.224471 11296 webserver.cc:460] Webserver started at http://127.0.0.1:37215/ using document root <none> and password file <none>
I20260806 19:47:49.225245 11296 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:49.225356 11296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:49.225615 11296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:49.228892 11296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "78dd2473e0ec48668e160e58ad2874c4"
format_stamp: "Formatted at 2026-08-06 19:47:49 on dist-test-slave-77v9"
server_key: "0be7e86ef811c10540ed2738cc23455d"
server_key_iv: "dcdab73de065215aceedeea9fa96b883"
server_key_version: "encryptionkey@0"
I20260806 19:47:49.229694 11296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "78dd2473e0ec48668e160e58ad2874c4"
format_stamp: "Formatted at 2026-08-06 19:47:49 on dist-test-slave-77v9"
server_key: "0be7e86ef811c10540ed2738cc23455d"
server_key_iv: "dcdab73de065215aceedeea9fa96b883"
server_key_version: "encryptionkey@0"
I20260806 19:47:49.235596 11296 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 19:47:49.239574 11311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:49.241322 11296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:47:49.241533 11296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "78dd2473e0ec48668e160e58ad2874c4"
format_stamp: "Formatted at 2026-08-06 19:47:49 on dist-test-slave-77v9"
server_key: "0be7e86ef811c10540ed2738cc23455d"
server_key_iv: "dcdab73de065215aceedeea9fa96b883"
server_key_version: "encryptionkey@0"
I20260806 19:47:49.241739 11296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:49.272578 11296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:49.277279 11296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:49.277623 11296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:49.297713 11296 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41629
I20260806 19:47:49.297753 11362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41629 every 8 connection(s)
I20260806 19:47:49.299240 11296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:49.303179 11363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:49.309312 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 11296
I20260806 19:47:49.309543 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:49.310052 22313 external_mini_cluster.cc:1428] Setting key 21cdc244d23beb2f6ac70d12e6096f77
I20260806 19:47:49.314354 11363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4: Bootstrap starting.
I20260806 19:47:49.318215 11363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:49.319334 11363 log.cc:824] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:49.322605 11363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4: No bootstrap required, opened a new log
Aug 06 19:47:49 dist-test-slave-77v9 krb5kdc[11283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 19:47:49.331501 11363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78dd2473e0ec48668e160e58ad2874c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41629 } }
I20260806 19:47:49.331913 11363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:49.332041 11363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78dd2473e0ec48668e160e58ad2874c4, State: Initialized, Role: FOLLOWER
I20260806 19:47:49.332682 11363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78dd2473e0ec48668e160e58ad2874c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41629 } }
I20260806 19:47:49.333019 11363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:49.333189 11363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:49.333396 11363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:49.334671 11366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:49.312032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:46311 (local address 127.0.0.1:41629)
0806 19:47:49.312974 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:49.312986 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:49.313032 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:47:49.314937 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:49.314959 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:49.315363 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:49.315699 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:49.317740 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:49.319155 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:49.320774 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:49.321514 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:49.321653 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:49.326650 (+ 4997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:49.326659 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:49.326674 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:49.326717 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:49.330068 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:49.330971 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:49.330977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:49.330984 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:49.331150 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:49.331630 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:49.331642 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:49.331644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:49.332314 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:49.332692 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:49.333223 (+ 531us) server_negotiation.cc:299] Negotiation successful
0806 19:47:49.333591 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":435,"threads_started":1}
I20260806 19:47:49.335958 11363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78dd2473e0ec48668e160e58ad2874c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41629 } }
I20260806 19:47:49.336481 11363 leader_election.cc:302] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78dd2473e0ec48668e160e58ad2874c4; no voters:
I20260806 19:47:49.337216 11363 leader_election.cc:288] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:49.337350 11368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:49.338653 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:47:49.338608 11368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [term 1 LEADER]: Becoming Leader. State: Replica: 78dd2473e0ec48668e160e58ad2874c4, State: Running, Role: LEADER
I20260806 19:47:49.339197 11368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78dd2473e0ec48668e160e58ad2874c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41629 } }
I20260806 19:47:49.339639 11363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:49.342176 11370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78dd2473e0ec48668e160e58ad2874c4. Latest consensus state: current_term: 1 leader_uuid: "78dd2473e0ec48668e160e58ad2874c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78dd2473e0ec48668e160e58ad2874c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41629 } } }
I20260806 19:47:49.342643 11370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:49.343343 11369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78dd2473e0ec48668e160e58ad2874c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78dd2473e0ec48668e160e58ad2874c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41629 } } }
I20260806 19:47:49.343835 11369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:49.355043 11374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:49.361779 11374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:49.382061 11366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:49.356554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:34791 (local address 127.0.0.1:41629)
0806 19:47:49.356773 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:49.356780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:49.357430 (+ 650us) server_negotiation.cc:407] Connection header received
0806 19:47:49.357652 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:49.357659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:49.357750 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:49.357864 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:49.360795 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:49.362482 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:49.365052 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:49.366023 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:49.366149 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:49.371414 (+ 5265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:49.371425 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:49.371433 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:49.371492 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:49.376475 (+ 4983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:49.378153 (+ 1678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:49.378164 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:49.378171 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:49.378528 (+ 357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:49.379368 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:49.379377 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:49.379382 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:49.379944 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:49.380276 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:49.381218 (+ 942us) server_negotiation.cc:299] Negotiation successful
0806 19:47:49.381507 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 19:47:49.387959 11374 catalog_manager.cc:1347] Generated new cluster ID: 6ff60c4d18bf4a96befea0e27ed6154d
I20260806 19:47:49.388083 11374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:49.432977 11374 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:49.435012 11374 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:49.445083 11374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78dd2473e0ec48668e160e58ad2874c4: Generated new TSK 0
I20260806 19:47:49.446112 11374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:49.461160 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11296
I20260806 19:47:49.462893 11358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:49.560297 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 11296
W20260806 19:47:49.580090 11281 connection.cc:535] client connection to 127.0.0.1:41629 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41629): Connection reset by peer (error 104)
2026-08-06T19:47:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2260 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:49 dist-test-slave-77v9 krb5kdc[11396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:49 dist-test-slave-77v9 krb5kdc[11396](info): set up 2 sockets
Aug 06 19:47:49 dist-test-slave-77v9 krb5kdc[11396](info): commencing operation
krb5kdc: starting...
W20260806 19:47:50.737440 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:50 dist-test-slave-77v9 krb5kdc[11396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:47:51.076475 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46281
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:51.440219 11409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:51.440594 11409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:51.440778 11409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:51.448625 11409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:51.448807 11409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:51.448930 11409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:51.449050 11409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:51.449147 11409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:51.460170 11409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:51.461750 11409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:51.463167 11409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:51.476521 11414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:51.476593 11415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:51.476598 11417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:51.477681 11409 server_base.cc:1034] running on GCE node
I20260806 19:47:51.478825 11409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:51.481040 11409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:51.482362 11409 hybrid_clock.cc:648] HybridClock initialized: now 1786045671482245 us; error 108 us; skew 500 ppm
I20260806 19:47:51.482836 11409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:51 dist-test-slave-77v9 krb5kdc[11396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045671, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:51.487102 11409 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:47:51.488695 11409 webserver.cc:460] Webserver started at http://127.0.0.1:35565/ using document root <none> and password file <none>
I20260806 19:47:51.489466 11409 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:51.489627 11409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:51.489957 11409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:51.493182 11409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "419167af8a9a492aa9a0e4372223ad75"
format_stamp: "Formatted at 2026-08-06 19:47:51 on dist-test-slave-77v9"
server_key: "88d9568e824dc6f631bc3d148650a1ee"
server_key_iv: "ecd5c6ff8fda51dd96052165db9b3246"
server_key_version: "encryptionkey@0"
I20260806 19:47:51.494009 11409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "419167af8a9a492aa9a0e4372223ad75"
format_stamp: "Formatted at 2026-08-06 19:47:51 on dist-test-slave-77v9"
server_key: "88d9568e824dc6f631bc3d148650a1ee"
server_key_iv: "ecd5c6ff8fda51dd96052165db9b3246"
server_key_version: "encryptionkey@0"
I20260806 19:47:51.499598 11409 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:47:51.503436 11424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:51.504987 11409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:47:51.505185 11409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "419167af8a9a492aa9a0e4372223ad75"
format_stamp: "Formatted at 2026-08-06 19:47:51 on dist-test-slave-77v9"
server_key: "88d9568e824dc6f631bc3d148650a1ee"
server_key_iv: "ecd5c6ff8fda51dd96052165db9b3246"
server_key_version: "encryptionkey@0"
I20260806 19:47:51.505416 11409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:51.517066 11409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:51.521935 11409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:51.522284 11409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:51.542680 11409 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46281
I20260806 19:47:51.542719 11475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46281 every 8 connection(s)
I20260806 19:47:51.544173 11409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:51.548373 11476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:51.550702 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 11409
I20260806 19:47:51.550899 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:51.551541 22313 external_mini_cluster.cc:1428] Setting key a2f37ca4a867ecdc1b96173eac7a8bc4
I20260806 19:47:51.563898 11476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75: Bootstrap starting.
I20260806 19:47:51.566134 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:51.553744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:55893 (local address 127.0.0.1:46281)
0806 19:47:51.554603 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:51.554615 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:51.554664 (+ 49us) server_negotiation.cc:407] Connection header received
0806 19:47:51.556279 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:51.556301 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:51.556731 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:51.557244 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:51.557865 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:51.559188 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:51.561196 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:51.561905 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:51.562043 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:51.563527 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:51.563536 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:51.563551 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:51.564218 (+ 667us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:51.564302 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:51.564329 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:51.564471 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:51.564813 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 19:47:51.565050 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":405,"threads_started":1}
I20260806 19:47:51.568181 11476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:51.569530 11476 log.cc:824] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:51.570684 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:47:51.573058 11476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75: No bootstrap required, opened a new log
I20260806 19:47:51.583446 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:51.576441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:59581 (local address 127.0.0.1:46281)
0806 19:47:51.576655 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:51.576663 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:51.577092 (+ 429us) server_negotiation.cc:407] Connection header received
0806 19:47:51.577260 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:51.577266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:51.577362 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:51.577477 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:51.577995 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:51.579179 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:51.580580 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:51.581351 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:51.581496 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:51.582623 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:51.582628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:51.582633 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:51.582654 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:51.582721 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:51.582742 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:51.582870 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:51.583051 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 19:47:51.583111 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 19:47:51.583328 11476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419167af8a9a492aa9a0e4372223ad75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46281 } }
I20260806 19:47:51.583871 11476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:51.583997 11476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 419167af8a9a492aa9a0e4372223ad75, State: Initialized, Role: FOLLOWER
I20260806 19:47:51.584744 11476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419167af8a9a492aa9a0e4372223ad75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46281 } }
I20260806 19:47:51.585090 11476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:51.585261 11476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:51.585441 11476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:51.587728 11476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419167af8a9a492aa9a0e4372223ad75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46281 } }
I20260806 19:47:51.588264 11476 leader_election.cc:302] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 419167af8a9a492aa9a0e4372223ad75; no voters:
I20260806 19:47:51.588915 11476 leader_election.cc:288] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:51.589051 11486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:51.590284 11486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [term 1 LEADER]: Becoming Leader. State: Replica: 419167af8a9a492aa9a0e4372223ad75, State: Running, Role: LEADER
I20260806 19:47:51.590916 11486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419167af8a9a492aa9a0e4372223ad75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46281 } }
I20260806 19:47:51.591217 11476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:51.594003 11488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 419167af8a9a492aa9a0e4372223ad75. Latest consensus state: current_term: 1 leader_uuid: "419167af8a9a492aa9a0e4372223ad75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419167af8a9a492aa9a0e4372223ad75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46281 } } }
I20260806 19:47:51.594125 11487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "419167af8a9a492aa9a0e4372223ad75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419167af8a9a492aa9a0e4372223ad75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46281 } } }
I20260806 19:47:51.594449 11488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:51.594519 11487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:51.595613 11495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:51.601462 11495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:51.611378 11495 catalog_manager.cc:1347] Generated new cluster ID: 077122c7808345a8976d867de543a4ca
I20260806 19:47:51.611515 11495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:51.655186 11495 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:51.656642 11495 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:51.673764 11495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 419167af8a9a492aa9a0e4372223ad75: Generated new TSK 0
I20260806 19:47:51.674736 11495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:51.685570 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11409
W20260806 19:47:51.687007 11471 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.120:55893
W20260806 19:47:51.687780 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11409: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:47:51.687939 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 11409
W20260806 19:47:51.707782 11394 connection.cc:535] client connection to 127.0.0.1:46281 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46281): Connection reset by peer (error 104)
2026-08-06T19:47:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2127 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:51 dist-test-slave-77v9 krb5kdc[11507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:51 dist-test-slave-77v9 krb5kdc[11507](info): set up 2 sockets
Aug 06 19:47:51 dist-test-slave-77v9 krb5kdc[11507](info): commencing operation
krb5kdc: starting...
W20260806 19:47:52.873708 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:53 dist-test-slave-77v9 krb5kdc[11507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:47:53.222538 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42525
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:53.580996 11520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:53.581372 11520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:53.581568 11520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:53.589608 11520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:53.589841 11520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:53.589968 11520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:53.590097 11520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:53.590196 11520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:53.601749 11520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:53.603313 11520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:53.604753 11520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:53.618162 11525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:53.618135 11526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:53.618089 11528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:53.619004 11520 server_base.cc:1034] running on GCE node
I20260806 19:47:53.619899 11520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:53.621537 11520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:53.622814 11520 hybrid_clock.cc:648] HybridClock initialized: now 1786045673622760 us; error 41 us; skew 500 ppm
I20260806 19:47:53.623219 11520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:53 dist-test-slave-77v9 krb5kdc[11507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045673, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:53.630326 11520 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:47:53.631950 11520 webserver.cc:460] Webserver started at http://127.0.0.1:44975/ using document root <none> and password file <none>
I20260806 19:47:53.632624 11520 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:53.632730 11520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:53.633061 11520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:53.636140 11520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "63a1004346154e0a90d92a63da66da6e"
format_stamp: "Formatted at 2026-08-06 19:47:53 on dist-test-slave-77v9"
server_key: "1f0f5ae4087d144e32bb2607625eb405"
server_key_iv: "c5bdf00ed8e7868fd49db84302d8aae2"
server_key_version: "encryptionkey@0"
I20260806 19:47:53.636970 11520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "63a1004346154e0a90d92a63da66da6e"
format_stamp: "Formatted at 2026-08-06 19:47:53 on dist-test-slave-77v9"
server_key: "1f0f5ae4087d144e32bb2607625eb405"
server_key_iv: "c5bdf00ed8e7868fd49db84302d8aae2"
server_key_version: "encryptionkey@0"
I20260806 19:47:53.642341 11520 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 19:47:53.646086 11535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:53.647615 11520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:47:53.647830 11520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "63a1004346154e0a90d92a63da66da6e"
format_stamp: "Formatted at 2026-08-06 19:47:53 on dist-test-slave-77v9"
server_key: "1f0f5ae4087d144e32bb2607625eb405"
server_key_iv: "c5bdf00ed8e7868fd49db84302d8aae2"
server_key_version: "encryptionkey@0"
I20260806 19:47:53.648023 11520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:53.654196 11520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:53.658782 11520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:53.659143 11520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:53.681232 11520 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42525
I20260806 19:47:53.681288 11586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42525 every 8 connection(s)
I20260806 19:47:53.682760 11520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:53.686861 11587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:53.686990 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 11520
I20260806 19:47:53.687165 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:53.687713 22313 external_mini_cluster.cc:1428] Setting key 352570ce22573e6418910c2d48749e2f
I20260806 19:47:53.697077 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:53.689798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:35435 (local address 127.0.0.1:42525)
0806 19:47:53.690831 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:53.690844 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:53.690893 (+ 49us) server_negotiation.cc:407] Connection header received
0806 19:47:53.693293 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:53.693322 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:53.693728 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:53.694105 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:53.695334 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:53.695341 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:53.695353 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:53.695369 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:53.695431 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:53.695457 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:53.695631 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:53.695756 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 19:47:53.695982 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":496,"threads_started":1}
I20260806 19:47:53.701036 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:47:53.701648 11587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e: Bootstrap starting.
I20260806 19:47:53.706359 11587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:53.707965 11587 log.cc:824] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:53.711552 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:53.707548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:39549 (local address 127.0.0.1:42525)
0806 19:47:53.707824 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:53.707830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:53.708212 (+ 382us) server_negotiation.cc:407] Connection header received
0806 19:47:53.708629 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:53.708667 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:53.708758 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:53.708994 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:53.710685 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:53.710693 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:53.710701 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:53.710725 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:53.710787 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:53.710818 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:53.710969 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:53.711183 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 19:47:53.711255 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 19:47:53.712088 11587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e: No bootstrap required, opened a new log
I20260806 19:47:53.719996 11587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a1004346154e0a90d92a63da66da6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42525 } }
I20260806 19:47:53.720432 11587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:53.720566 11587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a1004346154e0a90d92a63da66da6e, State: Initialized, Role: FOLLOWER
I20260806 19:47:53.721330 11587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a1004346154e0a90d92a63da66da6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42525 } }
I20260806 19:47:53.721648 11587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:53.721805 11587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:53.722043 11587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:53.724361 11587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a1004346154e0a90d92a63da66da6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42525 } }
I20260806 19:47:53.724970 11587 leader_election.cc:302] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63a1004346154e0a90d92a63da66da6e; no voters:
I20260806 19:47:53.725621 11587 leader_election.cc:288] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:53.725852 11597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:53.726598 11597 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [term 1 LEADER]: Becoming Leader. State: Replica: 63a1004346154e0a90d92a63da66da6e, State: Running, Role: LEADER
I20260806 19:47:53.727205 11597 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a1004346154e0a90d92a63da66da6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42525 } }
I20260806 19:47:53.727941 11587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:53.729990 11598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63a1004346154e0a90d92a63da66da6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a1004346154e0a90d92a63da66da6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42525 } } }
I20260806 19:47:53.730556 11598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:53.730553 11599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63a1004346154e0a90d92a63da66da6e. Latest consensus state: current_term: 1 leader_uuid: "63a1004346154e0a90d92a63da66da6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a1004346154e0a90d92a63da66da6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42525 } } }
I20260806 19:47:53.730924 11599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:53.731630 11604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:53.738054 11604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:53.747915 11604 catalog_manager.cc:1347] Generated new cluster ID: ce9c28e743e74558bfa9c411da1ce507
I20260806 19:47:53.748095 11604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:53.766978 11604 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:53.767729 11604 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:53.780649 11604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63a1004346154e0a90d92a63da66da6e: Generated new TSK 0
I20260806 19:47:53.781625 11604 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:53.793851 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11520
W20260806 19:47:53.795022 11582 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.120:35435
W20260806 19:47:53.795795 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11520: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:47:53.795954 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 11520
2026-08-06T19:47:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2106 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:53 dist-test-slave-77v9 krb5kdc[11618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:53 dist-test-slave-77v9 krb5kdc[11618](info): set up 2 sockets
Aug 06 19:47:53 dist-test-slave-77v9 krb5kdc[11618](info): commencing operation
krb5kdc: starting...
W20260806 19:47:54.986814 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.002s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:55 dist-test-slave-77v9 krb5kdc[11618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:47:55.330758 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37781
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:55.693107 11631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:55.693430 11631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:55.693606 11631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:55.701071 11631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:55.701191 11631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:55.701352 11631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:55.701438 11631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:55.701524 11631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:55.711750 11631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37781
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:55.713299 11631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:55.714735 11631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:55.727596 11636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:55.727627 11637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:55.727564 11639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:55.728543 11631 server_base.cc:1034] running on GCE node
I20260806 19:47:55.729496 11631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:55.731096 11631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:55.732329 11631 hybrid_clock.cc:648] HybridClock initialized: now 1786045675732269 us; error 53 us; skew 500 ppm
I20260806 19:47:55.732744 11631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:55 dist-test-slave-77v9 krb5kdc[11618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045675, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:55.736378 11631 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:47:55.738138 11631 webserver.cc:460] Webserver started at http://127.0.0.1:44841/ using document root <none> and password file <none>
I20260806 19:47:55.738870 11631 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:55.739037 11631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:55.739364 11631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:55.742600 11631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "6fa6dfbcdf7d49cc835dd78adf9d3814"
format_stamp: "Formatted at 2026-08-06 19:47:55 on dist-test-slave-77v9"
server_key: "3122e3553f24f9e0b957d20e823c0659"
server_key_iv: "842e64dca726da83719ef31eba8d8b64"
server_key_version: "encryptionkey@0"
I20260806 19:47:55.743432 11631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "6fa6dfbcdf7d49cc835dd78adf9d3814"
format_stamp: "Formatted at 2026-08-06 19:47:55 on dist-test-slave-77v9"
server_key: "3122e3553f24f9e0b957d20e823c0659"
server_key_iv: "842e64dca726da83719ef31eba8d8b64"
server_key_version: "encryptionkey@0"
I20260806 19:47:55.749176 11631 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 19:47:55.753382 11646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:55.755059 11631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:47:55.755280 11631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "6fa6dfbcdf7d49cc835dd78adf9d3814"
format_stamp: "Formatted at 2026-08-06 19:47:55 on dist-test-slave-77v9"
server_key: "3122e3553f24f9e0b957d20e823c0659"
server_key_iv: "842e64dca726da83719ef31eba8d8b64"
server_key_version: "encryptionkey@0"
I20260806 19:47:55.755514 11631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:55.770589 11631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:55.775158 11631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:55.775513 11631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:55.795991 11631 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37781
I20260806 19:47:55.796030 11697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37781 every 8 connection(s)
I20260806 19:47:55.797843 11631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:55.802207 11698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:55.805665 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 11631
I20260806 19:47:55.805871 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:55.806474 22313 external_mini_cluster.cc:1428] Setting key 1b08c97f150ed3ca937df824a8162c73
I20260806 19:47:55.816917 11698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814: Bootstrap starting.
I20260806 19:47:55.821436 11698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:55.823196 11698 log.cc:824] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814: Log is configured to *not* fsync() on all Append() calls
Aug 06 19:47:55 dist-test-slave-77v9 krb5kdc[11618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 19:47:55.827436 11698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814: No bootstrap required, opened a new log
I20260806 19:47:55.836300 11701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:55.808661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:37003 (local address 127.0.0.1:37781)
0806 19:47:55.809699 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:55.809712 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:55.809760 (+ 48us) server_negotiation.cc:407] Connection header received
0806 19:47:55.811881 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:55.811909 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:55.812412 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:55.812890 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:55.815016 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:55.816484 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:55.818634 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:55.819401 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:55.819582 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:55.826376 (+ 6794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:55.826385 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:55.826403 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:55.826446 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:55.830873 (+ 4427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:55.832122 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:55.832129 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:55.832137 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:55.832324 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:55.833021 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:55.833026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:55.833028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:55.833703 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:55.834009 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:55.834596 (+ 587us) server_negotiation.cc:299] Negotiation successful
0806 19:47:55.835033 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":439,"threads_started":1}
I20260806 19:47:55.837966 11698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa6dfbcdf7d49cc835dd78adf9d3814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37781 } }
I20260806 19:47:55.838527 11698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:55.838649 11698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fa6dfbcdf7d49cc835dd78adf9d3814, State: Initialized, Role: FOLLOWER
I20260806 19:47:55.839346 11698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa6dfbcdf7d49cc835dd78adf9d3814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37781 } }
I20260806 19:47:55.839732 11698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:55.839893 11698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:55.840101 11698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:55.841225 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:47:55.843053 11698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa6dfbcdf7d49cc835dd78adf9d3814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37781 } }
I20260806 19:47:55.843626 11698 leader_election.cc:302] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fa6dfbcdf7d49cc835dd78adf9d3814; no voters:
I20260806 19:47:55.844388 11698 leader_election.cc:288] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:55.844551 11705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:55.846033 11705 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [term 1 LEADER]: Becoming Leader. State: Replica: 6fa6dfbcdf7d49cc835dd78adf9d3814, State: Running, Role: LEADER
I20260806 19:47:55.846846 11705 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa6dfbcdf7d49cc835dd78adf9d3814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37781 } }
I20260806 19:47:55.847153 11698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:55.850167 11708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fa6dfbcdf7d49cc835dd78adf9d3814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa6dfbcdf7d49cc835dd78adf9d3814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37781 } } }
I20260806 19:47:55.850548 11708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:55.849849 11709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fa6dfbcdf7d49cc835dd78adf9d3814. Latest consensus state: current_term: 1 leader_uuid: "6fa6dfbcdf7d49cc835dd78adf9d3814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa6dfbcdf7d49cc835dd78adf9d3814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37781 } } }
I20260806 19:47:55.862965 11709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814 [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:55.870818 11713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:55.879942 11713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:55.881500 11701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:55.846497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:58145 (local address 127.0.0.1:37781)
0806 19:47:55.846931 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:55.846939 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:55.847537 (+ 598us) server_negotiation.cc:407] Connection header received
0806 19:47:55.847761 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:55.847766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:55.847855 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:55.847968 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:55.850622 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:55.863479 (+ 12857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:55.866337 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:55.867322 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:55.867485 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:55.872447 (+ 4962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:55.872453 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:55.872456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:47:55.872494 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:55.876330 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:55.877820 (+ 1490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:55.877828 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:55.877837 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:55.878009 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:47:55.878725 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:47:55.878731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:47:55.878734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:47:55.879195 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:55.879517 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:55.880859 (+ 1342us) server_negotiation.cc:299] Negotiation successful
0806 19:47:55.881064 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":286}
I20260806 19:47:55.892011 11713 catalog_manager.cc:1347] Generated new cluster ID: fe7fe2022ad24900b0644568dd65b75e
I20260806 19:47:55.892151 11713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:55.925573 11713 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:55.927536 11713 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:55.940308 11713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fa6dfbcdf7d49cc835dd78adf9d3814: Generated new TSK 0
I20260806 19:47:55.941407 11713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:55.953922 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11631
I20260806 19:47:55.955768 11693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:47:56.055776 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 11631
W20260806 19:47:56.075603 11616 connection.cc:535] client connection to 127.0.0.1:37781 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37781): Connection reset by peer (error 104)
2026-08-06T19:47:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2260 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:56 dist-test-slave-77v9 krb5kdc[11731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:56 dist-test-slave-77v9 krb5kdc[11731](info): set up 2 sockets
Aug 06 19:47:56 dist-test-slave-77v9 krb5kdc[11731](info): commencing operation
krb5kdc: starting...
W20260806 19:47:57.261474 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:57 dist-test-slave-77v9 krb5kdc[11731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:47:57.601687 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:47:57.959621 11744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:47:57.959985 11744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:47:57.960083 11744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:47:57.967514 11744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:47:57.967643 11744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:47:57.967690 11744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:47:57.967748 11744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:47:57.967794 11744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:47:57.978482 11744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36323
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:47:57.979817 11744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:47:57.981145 11744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:47:57.994338 11750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:57.994344 11749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:47:57.994874 11752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:47:57.995157 11744 server_base.cc:1034] running on GCE node
I20260806 19:47:57.996016 11744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:47:57.997746 11744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:47:57.999001 11744 hybrid_clock.cc:648] HybridClock initialized: now 1786045677998938 us; error 69 us; skew 500 ppm
I20260806 19:47:57.999455 11744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:47:57 dist-test-slave-77v9 krb5kdc[11731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045677, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:47:58.003029 11744 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:47:58.004752 11744 webserver.cc:460] Webserver started at http://127.0.0.1:37665/ using document root <none> and password file <none>
I20260806 19:47:58.005573 11744 fs_manager.cc:359] Metadata directory not provided
I20260806 19:47:58.005740 11744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:47:58.006089 11744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:47:58.009572 11744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "60a5297f0b06453eb5df828e19aae1dd"
format_stamp: "Formatted at 2026-08-06 19:47:58 on dist-test-slave-77v9"
server_key: "584d9128791528124c05b09257c6008f"
server_key_iv: "a41188bff2df099b11b55d0381e1e961"
server_key_version: "encryptionkey@0"
I20260806 19:47:58.010488 11744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "60a5297f0b06453eb5df828e19aae1dd"
format_stamp: "Formatted at 2026-08-06 19:47:58 on dist-test-slave-77v9"
server_key: "584d9128791528124c05b09257c6008f"
server_key_iv: "a41188bff2df099b11b55d0381e1e961"
server_key_version: "encryptionkey@0"
I20260806 19:47:58.016642 11744 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 19:47:58.020714 11759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:47:58.022289 11744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:47:58.022531 11744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "60a5297f0b06453eb5df828e19aae1dd"
format_stamp: "Formatted at 2026-08-06 19:47:58 on dist-test-slave-77v9"
server_key: "584d9128791528124c05b09257c6008f"
server_key_iv: "a41188bff2df099b11b55d0381e1e961"
server_key_version: "encryptionkey@0"
I20260806 19:47:58.022773 11744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:47:58.038514 11744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:47:58.043789 11744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:47:58.044134 11744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:47:58.064539 11744 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36323
I20260806 19:47:58.064561 11810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36323 every 8 connection(s)
I20260806 19:47:58.066205 11744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:47:58.070472 11811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:47:58.076278 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 11744
I20260806 19:47:58.076508 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:47:58.077076 22313 external_mini_cluster.cc:1428] Setting key 7267bb02533f0238662f9ab87dec2aa5
I20260806 19:47:58.081981 11811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd: Bootstrap starting.
I20260806 19:47:58.085845 11811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd: Neither blocks nor log segments found. Creating new log.
I20260806 19:47:58.087206 11811 log.cc:824] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd: Log is configured to *not* fsync() on all Append() calls
I20260806 19:47:58.090971 11811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd: No bootstrap required, opened a new log
I20260806 19:47:58.092214 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:58.079133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:33119 (local address 127.0.0.1:36323)
0806 19:47:58.080043 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:58.080055 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:58.080099 (+ 44us) server_negotiation.cc:407] Connection header received
0806 19:47:58.081920 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:58.081945 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:58.082403 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:58.082742 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:58.083427 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:58.084783 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:58.086889 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:58.087772 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:58.087996 (+ 224us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:58.089273 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:58.089284 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:58.089303 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:58.090057 (+ 754us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:58.090171 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:58.090204 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:58.090381 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:58.090679 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 19:47:58.090868 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":440,"threads_started":1}
I20260806 19:47:58.096849 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:47:58.100590 11811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a5297f0b06453eb5df828e19aae1dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36323 } }
I20260806 19:47:58.101104 11811 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:47:58.101218 11811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60a5297f0b06453eb5df828e19aae1dd, State: Initialized, Role: FOLLOWER
I20260806 19:47:58.101999 11811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a5297f0b06453eb5df828e19aae1dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36323 } }
I20260806 19:47:58.102468 11811 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:47:58.102646 11811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:47:58.102882 11811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:47:58.106192 11811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a5297f0b06453eb5df828e19aae1dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36323 } }
I20260806 19:47:58.106880 11811 leader_election.cc:302] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60a5297f0b06453eb5df828e19aae1dd; no voters:
I20260806 19:47:58.107633 11811 leader_election.cc:288] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:47:58.108114 11821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:47:58.109062 11821 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [term 1 LEADER]: Becoming Leader. State: Replica: 60a5297f0b06453eb5df828e19aae1dd, State: Running, Role: LEADER
I20260806 19:47:58.109851 11821 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a5297f0b06453eb5df828e19aae1dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36323 } }
I20260806 19:47:58.110671 11811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:47:58.110715 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:47:58.101873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:51795 (local address 127.0.0.1:36323)
0806 19:47:58.102311 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 19:47:58.102318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:47:58.103057 (+ 739us) server_negotiation.cc:407] Connection header received
0806 19:47:58.103280 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:47:58.103285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:47:58.103367 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:47:58.103477 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:47:58.104119 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:58.105043 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:58.107003 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:47:58.107899 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:47:58.108074 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:47:58.109666 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:47:58.109676 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:47:58.109685 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:47:58.109715 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:47:58.109767 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:47:58.109791 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:47:58.109933 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 19:47:58.110178 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 19:47:58.110321 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 19:47:58.113481 11822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60a5297f0b06453eb5df828e19aae1dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a5297f0b06453eb5df828e19aae1dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36323 } } }
I20260806 19:47:58.113883 11822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:58.113600 11823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60a5297f0b06453eb5df828e19aae1dd. Latest consensus state: current_term: 1 leader_uuid: "60a5297f0b06453eb5df828e19aae1dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a5297f0b06453eb5df828e19aae1dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36323 } } }
I20260806 19:47:58.114118 11823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd [sys.catalog]: This master's current role is: LEADER
I20260806 19:47:58.114876 11829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:47:58.123641 11829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:47:58.133679 11829 catalog_manager.cc:1347] Generated new cluster ID: 04ede09b7fdd4da392c1beb5c0e49629
I20260806 19:47:58.133882 11829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:47:58.155005 11829 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:47:58.156430 11829 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:47:58.166188 11829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60a5297f0b06453eb5df828e19aae1dd: Generated new TSK 0
I20260806 19:47:58.167141 11829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:47:58.179558 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11744
W20260806 19:47:58.180855 11806 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.120:33119
W20260806 19:47:58.181717 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11744: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:47:58.181872 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 11744
W20260806 19:47:58.201260 11729 connection.cc:535] client connection to 127.0.0.1:36323 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36323): Connection reset by peer (error 104)
2026-08-06T19:47:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2125 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:47:58 dist-test-slave-77v9 krb5kdc[11842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:47:58 dist-test-slave-77v9 krb5kdc[11842](info): set up 2 sockets
Aug 06 19:47:58 dist-test-slave-77v9 krb5kdc[11842](info): commencing operation
krb5kdc: starting...
W20260806 19:47:59.369478 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.010s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:47:59 dist-test-slave-77v9 krb5kdc[11842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:47:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:47:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:47:59.718326 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45107
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:48:00.074757 11855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:48:00.075076 11855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:48:00.075173 11855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:48:00.082840 11855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:48:00.082976 11855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:48:00.083024 11855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:48:00.083088 11855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:48:00.083133 11855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:48:00.093756 11855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45107
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:48:00.095212 11855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:48:00.096655 11855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:48:00.109664 11860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:00.109601 11862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:00.109637 11864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:00.110919 11855 server_base.cc:1034] running on GCE node
I20260806 19:48:00.111795 11855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:48:00.113415 11855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:48:00.114702 11855 hybrid_clock.cc:648] HybridClock initialized: now 1786045680114648 us; error 45 us; skew 500 ppm
I20260806 19:48:00.115120 11855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:48:00 dist-test-slave-77v9 krb5kdc[11842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045680, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:48:00.122058 11855 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:48:00.123894 11855 webserver.cc:460] Webserver started at http://127.0.0.1:33251/ using document root <none> and password file <none>
I20260806 19:48:00.124698 11855 fs_manager.cc:359] Metadata directory not provided
I20260806 19:48:00.124908 11855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:48:00.125291 11855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:48:00.128762 11855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "da4380bc8c684982859f9fde8a1b6768"
format_stamp: "Formatted at 2026-08-06 19:48:00 on dist-test-slave-77v9"
server_key: "84a4d52a3ece928d16fdfad24d36e2d3"
server_key_iv: "138dd65a965e20a21fd1c9d308973bd0"
server_key_version: "encryptionkey@0"
I20260806 19:48:00.129752 11855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "da4380bc8c684982859f9fde8a1b6768"
format_stamp: "Formatted at 2026-08-06 19:48:00 on dist-test-slave-77v9"
server_key: "84a4d52a3ece928d16fdfad24d36e2d3"
server_key_iv: "138dd65a965e20a21fd1c9d308973bd0"
server_key_version: "encryptionkey@0"
I20260806 19:48:00.135804 11855 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:48:00.139766 11871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:48:00.141646 11855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:48:00.141870 11855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "da4380bc8c684982859f9fde8a1b6768"
format_stamp: "Formatted at 2026-08-06 19:48:00 on dist-test-slave-77v9"
server_key: "84a4d52a3ece928d16fdfad24d36e2d3"
server_key_iv: "138dd65a965e20a21fd1c9d308973bd0"
server_key_version: "encryptionkey@0"
I20260806 19:48:00.142109 11855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:48:00.148142 11855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:48:00.152738 11855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:48:00.153132 11855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:48:00.173517 11855 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45107
I20260806 19:48:00.173557 11922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45107 every 8 connection(s)
I20260806 19:48:00.175172 11855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:48:00.179278 11923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:48:00.182435 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 11855
I20260806 19:48:00.182662 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:48:00.183164 22313 external_mini_cluster.cc:1428] Setting key ae8eff0014e4b8a73cd7d0f8671cc8f9
I20260806 19:48:00.192729 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:00.185400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:48617 (local address 127.0.0.1:45107)
0806 19:48:00.186228 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:00.186245 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:00.186297 (+ 52us) server_negotiation.cc:407] Connection header received
0806 19:48:00.188568 (+ 2271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:00.188604 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:00.189133 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:00.189622 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:00.190699 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:00.190708 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:00.190724 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:48:00.190749 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:00.190825 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:48:00.190851 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:00.191104 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:00.191246 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 19:48:00.191444 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":328,"threads_started":1}
I20260806 19:48:00.193650 11923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768: Bootstrap starting.
I20260806 19:48:00.197343 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:48:00.197383 11923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768: Neither blocks nor log segments found. Creating new log.
I20260806 19:48:00.198665 11923 log.cc:824] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768: Log is configured to *not* fsync() on all Append() calls
I20260806 19:48:00.202445 11923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768: No bootstrap required, opened a new log
I20260806 19:48:00.206403 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:00.203466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.120:45845 (local address 127.0.0.1:45107)
0806 19:48:00.203661 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:00.203666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:00.204130 (+ 464us) server_negotiation.cc:407] Connection header received
0806 19:48:00.204276 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:00.204282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:00.204353 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:00.204464 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:00.205615 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:00.205621 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:00.205627 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:48:00.205646 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:00.205694 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:48:00.205720 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:00.205850 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:00.206009 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 19:48:00.206082 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 19:48:00.211185 11923 raft_consensus.cc:348] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da4380bc8c684982859f9fde8a1b6768" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260806 19:48:00.211572 11923 raft_consensus.cc:374] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:48:00.211650 11923 raft_consensus.cc:729] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da4380bc8c684982859f9fde8a1b6768, State: Initialized, Role: FOLLOWER
I20260806 19:48:00.212271 11923 consensus_queue.cc:260] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da4380bc8c684982859f9fde8a1b6768" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260806 19:48:00.212515 11923 raft_consensus.cc:388] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:48:00.212611 11923 raft_consensus.cc:482] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:48:00.212736 11923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:48:00.215500 11923 raft_consensus.cc:504] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da4380bc8c684982859f9fde8a1b6768" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260806 19:48:00.216037 11923 leader_election.cc:302] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da4380bc8c684982859f9fde8a1b6768; no voters:
I20260806 19:48:00.216681 11923 leader_election.cc:288] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:48:00.216883 11933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:48:00.218166 11933 raft_consensus.cc:686] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [term 1 LEADER]: Becoming Leader. State: Replica: da4380bc8c684982859f9fde8a1b6768, State: Running, Role: LEADER
I20260806 19:48:00.218786 11933 consensus_queue.cc:237] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da4380bc8c684982859f9fde8a1b6768" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260806 19:48:00.219066 11923 sys_catalog.cc:564] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:48:00.222000 11935 sys_catalog.cc:455] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da4380bc8c684982859f9fde8a1b6768" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da4380bc8c684982859f9fde8a1b6768" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } } }
I20260806 19:48:00.222365 11935 sys_catalog.cc:458] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:00.222828 11936 sys_catalog.cc:455] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da4380bc8c684982859f9fde8a1b6768. Latest consensus state: current_term: 1 leader_uuid: "da4380bc8c684982859f9fde8a1b6768" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da4380bc8c684982859f9fde8a1b6768" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } } }
I20260806 19:48:00.223129 11936 sys_catalog.cc:458] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768 [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:00.225528 11943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:48:00.231287 11943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:48:00.240363 11943 catalog_manager.cc:1347] Generated new cluster ID: f80b3ce352634d19a2356b327e3fc04b
I20260806 19:48:00.240491 11943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:48:00.284155 11943 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:48:00.284670 11943 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:48:00.294164 11943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da4380bc8c684982859f9fde8a1b6768: Generated new TSK 0
I20260806 19:48:00.295092 11943 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:48:00.306084 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11855
W20260806 19:48:00.307349 11918 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.120:48617
W20260806 19:48:00.308193 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11855: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:48:00.308387 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 11855
2026-08-06T19:48:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2126 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:48:00 dist-test-slave-77v9 krb5kdc[11955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:48:00 dist-test-slave-77v9 krb5kdc[11955](info): set up 2 sockets
Aug 06 19:48:00 dist-test-slave-77v9 krb5kdc[11955](info): commencing operation
krb5kdc: starting...
W20260806 19:48:01.503152 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:48:01 dist-test-slave-77v9 krb5kdc[11955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:48:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:48:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:48:01.845211 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42649
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:42649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:48:02.205420 11968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:48:02.205724 11968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:48:02.205818 11968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:48:02.213058 11968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:48:02.213179 11968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:48:02.213227 11968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:48:02.213289 11968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:48:02.213328 11968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:48:02.223280 11968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:42649
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:42649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:48:02.224611 11968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:48:02.225903 11968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:48:02.238087 11973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:02.238132 11974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:02.238173 11976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:02.239087 11968 server_base.cc:1034] running on GCE node
I20260806 19:48:02.239948 11968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:48:02.241510 11968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:48:02.242756 11968 hybrid_clock.cc:648] HybridClock initialized: now 1786045682242684 us; error 68 us; skew 500 ppm
I20260806 19:48:02.243163 11968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:48:02 dist-test-slave-77v9 krb5kdc[11955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045682, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:48:02.246446 11968 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:48:02.248159 11968 webserver.cc:460] Webserver started at http://127.21.202.126:37763/ using document root <none> and password file <none>
I20260806 19:48:02.248962 11968 fs_manager.cc:359] Metadata directory not provided
I20260806 19:48:02.249122 11968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:48:02.249431 11968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:48:02.252488 11968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "d8604614241e4f9e8360922c3531c480"
format_stamp: "Formatted at 2026-08-06 19:48:02 on dist-test-slave-77v9"
server_key: "901b54dbc72f465330a0dc1a4f4e6503"
server_key_iv: "b2e11e8e9f9f41ec01a60a975c62c2c8"
server_key_version: "encryptionkey@0"
I20260806 19:48:02.253374 11968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "d8604614241e4f9e8360922c3531c480"
format_stamp: "Formatted at 2026-08-06 19:48:02 on dist-test-slave-77v9"
server_key: "901b54dbc72f465330a0dc1a4f4e6503"
server_key_iv: "b2e11e8e9f9f41ec01a60a975c62c2c8"
server_key_version: "encryptionkey@0"
I20260806 19:48:02.259181 11968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:48:02.262976 11983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:48:02.264496 11968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 19:48:02.264726 11968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "d8604614241e4f9e8360922c3531c480"
format_stamp: "Formatted at 2026-08-06 19:48:02 on dist-test-slave-77v9"
server_key: "901b54dbc72f465330a0dc1a4f4e6503"
server_key_iv: "b2e11e8e9f9f41ec01a60a975c62c2c8"
server_key_version: "encryptionkey@0"
I20260806 19:48:02.265008 11968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:48:02.343119 11968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:48:02.348296 11968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:48:02.348652 11968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:48:02.370743 11968 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:42649
I20260806 19:48:02.370798 12035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:42649 every 8 connection(s)
I20260806 19:48:02.372299 11968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:48:02.376353 12036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:48:02.380301 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 11968
I20260806 19:48:02.380522 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:48:02.381150 22313 external_mini_cluster.cc:1428] Setting key ba317ef1ed056c791a8af63065644f29
I20260806 19:48:02.389636 12036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480: Bootstrap starting.
I20260806 19:48:02.392894 12036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480: Neither blocks nor log segments found. Creating new log.
I20260806 19:48:02.393970 12036 log.cc:824] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480: Log is configured to *not* fsync() on all Append() calls
Aug 06 19:48:02 dist-test-slave-77v9 krb5kdc[11955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:48:02.396956 12036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480: No bootstrap required, opened a new log
I20260806 19:48:02.406688 12039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:02.383418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42841 (local address 127.21.202.126:42649)
0806 19:48:02.384249 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:02.384261 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:02.384304 (+ 43us) server_negotiation.cc:407] Connection header received
0806 19:48:02.385994 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:02.386018 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:02.386436 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:02.386861 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:02.388366 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:02.389907 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:02.391168 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:02.391839 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:02.392041 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:02.396425 (+ 4384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:02.396434 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:02.396448 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:02.396490 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:02.401032 (+ 4542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:02.402023 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:02.402032 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:02.402042 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:02.402297 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:02.402824 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:02.402831 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:02.402834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:02.403676 (+ 842us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:02.404051 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:02.404625 (+ 574us) server_negotiation.cc:299] Negotiation successful
0806 19:48:02.405051 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":391,"threads_started":1}
I20260806 19:48:02.408654 12036 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8604614241e4f9e8360922c3531c480" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42649 } }
I20260806 19:48:02.409121 12036 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:48:02.409219 12036 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8604614241e4f9e8360922c3531c480, State: Initialized, Role: FOLLOWER
I20260806 19:48:02.409870 12036 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8604614241e4f9e8360922c3531c480" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42649 } }
I20260806 19:48:02.410198 12036 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:48:02.410406 12036 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:48:02.410620 12036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:48:02.411332 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:48:02.413282 12036 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8604614241e4f9e8360922c3531c480" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42649 } }
I20260806 19:48:02.413847 12036 leader_election.cc:302] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8604614241e4f9e8360922c3531c480; no voters:
I20260806 19:48:02.414565 12036 leader_election.cc:288] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:48:02.414678 12042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:48:02.416324 12042 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [term 1 LEADER]: Becoming Leader. State: Replica: d8604614241e4f9e8360922c3531c480, State: Running, Role: LEADER
I20260806 19:48:02.417284 12042 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8604614241e4f9e8360922c3531c480" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42649 } }
I20260806 19:48:02.417510 12036 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:48:02.422897 12046 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8604614241e4f9e8360922c3531c480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8604614241e4f9e8360922c3531c480" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42649 } } }
I20260806 19:48:02.423317 12046 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:02.423947 12047 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8604614241e4f9e8360922c3531c480. Latest consensus state: current_term: 1 leader_uuid: "d8604614241e4f9e8360922c3531c480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8604614241e4f9e8360922c3531c480" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 42649 } } }
I20260806 19:48:02.424356 12047 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480 [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:02.426158 12058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:48:02.432132 12058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:48:02.439323 12039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:02.416905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56755 (local address 127.21.202.126:42649)
0806 19:48:02.417084 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:02.417091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:02.417344 (+ 253us) server_negotiation.cc:407] Connection header received
0806 19:48:02.417569 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:02.417575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:02.417654 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:02.417780 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:02.419837 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:02.421241 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:02.425086 (+ 3845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:02.426181 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:02.426372 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:02.430126 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:02.430131 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:02.430135 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:02.430169 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:02.433913 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:02.435213 (+ 1300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:02.435223 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:02.435230 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:02.435450 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:02.436289 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:02.436297 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:02.436304 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:02.436986 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:02.437372 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:02.438395 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0806 19:48:02.438674 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":41}
I20260806 19:48:02.446465 12058 catalog_manager.cc:1347] Generated new cluster ID: c5e33f0587f6418c9c25e3ddad937170
I20260806 19:48:02.446667 12058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:48:02.463027 12058 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:48:02.464311 12058 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:48:02.472391 12058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8604614241e4f9e8360922c3531c480: Generated new TSK 0
I20260806 19:48:02.473613 12058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:48:02.485535 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 11968
I20260806 19:48:02.487073 12031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:48:02.581046 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 11968
W20260806 19:48:02.600399 11953 connection.cc:535] client connection to 127.21.202.126:42649 recv error: Network error: failed to read from TLS socket (remote: 127.21.202.126:42649): Connection reset by peer (error 104)
2026-08-06T19:48:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2272 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:48:02 dist-test-slave-77v9 krb5kdc[12068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:48:02 dist-test-slave-77v9 krb5kdc[12068](info): set up 2 sockets
Aug 06 19:48:02 dist-test-slave-77v9 krb5kdc[12068](info): commencing operation
krb5kdc: starting...
W20260806 19:48:03.757757 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:48:03 dist-test-slave-77v9 krb5kdc[12068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:48:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:48:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:48:04.097446 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:36085
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:36003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:36085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:48:04.455821 12081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:48:04.456142 12081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:48:04.456238 12081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:48:04.464147 12081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:48:04.464293 12081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:48:04.464342 12081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:48:04.464412 12081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:48:04.464458 12081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:48:04.474865 12081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:36003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:36085
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:36085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:48:04.476181 12081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:48:04.477626 12081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:48:04.489912 12086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:04.489950 12087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:04.489969 12089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:04.491528 12081 server_base.cc:1034] running on GCE node
I20260806 19:48:04.492359 12081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:48:04.493953 12081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:48:04.495168 12081 hybrid_clock.cc:648] HybridClock initialized: now 1786045684495096 us; error 70 us; skew 500 ppm
I20260806 19:48:04.495543 12081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:48:04 dist-test-slave-77v9 krb5kdc[12068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045684, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:48:04.498876 12081 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:48:04.500545 12081 webserver.cc:460] Webserver started at http://127.21.202.126:34671/ using document root <none> and password file <none>
I20260806 19:48:04.501358 12081 fs_manager.cc:359] Metadata directory not provided
I20260806 19:48:04.501515 12081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:48:04.501837 12081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:48:04.505127 12081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "37db17e6e91b4e34a54d44c0fdafc2ef"
format_stamp: "Formatted at 2026-08-06 19:48:04 on dist-test-slave-77v9"
server_key: "1ecd818b252c43400a883c4c40b9dafc"
server_key_iv: "ccbe5694d7d7781619ddf30bb28febc9"
server_key_version: "encryptionkey@0"
I20260806 19:48:04.505918 12081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "37db17e6e91b4e34a54d44c0fdafc2ef"
format_stamp: "Formatted at 2026-08-06 19:48:04 on dist-test-slave-77v9"
server_key: "1ecd818b252c43400a883c4c40b9dafc"
server_key_iv: "ccbe5694d7d7781619ddf30bb28febc9"
server_key_version: "encryptionkey@0"
I20260806 19:48:04.511572 12081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 19:48:04.515484 12096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:48:04.517179 12081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 19:48:04.517400 12081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "37db17e6e91b4e34a54d44c0fdafc2ef"
format_stamp: "Formatted at 2026-08-06 19:48:04 on dist-test-slave-77v9"
server_key: "1ecd818b252c43400a883c4c40b9dafc"
server_key_iv: "ccbe5694d7d7781619ddf30bb28febc9"
server_key_version: "encryptionkey@0"
I20260806 19:48:04.517639 12081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:48:04.545094 12081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:48:04.549525 12081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:48:04.549832 12081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:48:04.570382 12081 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:36085
I20260806 19:48:04.570426 12147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:36085 every 8 connection(s)
I20260806 19:48:04.571820 12081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:48:04.575709 12148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:48:04.581872 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 12081
I20260806 19:48:04.582075 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:48:04.582672 22313 external_mini_cluster.cc:1428] Setting key 34e7aba10f06696a20a216666a93f0d6
I20260806 19:48:04.587332 12148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef: Bootstrap starting.
I20260806 19:48:04.591074 12148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef: Neither blocks nor log segments found. Creating new log.
I20260806 19:48:04.592571 12148 log.cc:824] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef: Log is configured to *not* fsync() on all Append() calls
I20260806 19:48:04.596457 12148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef: No bootstrap required, opened a new log
I20260806 19:48:04.598869 12151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:04.584798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52671 (local address 127.21.202.126:36085)
0806 19:48:04.585826 (+ 1028us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:04.585839 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:04.585885 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:48:04.587819 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:04.587847 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:04.588278 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:04.588622 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:04.589367 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:04.590856 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:04.593357 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:04.594617 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:04.594833 (+ 216us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:04.596234 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:04.596241 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:04.596257 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:48:04.596949 (+ 692us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:04.597043 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:48:04.597075 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:04.597254 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:04.597549 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 19:48:04.597720 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":476,"threads_started":1}
I20260806 19:48:04.603006 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:48:04.605960 12148 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37db17e6e91b4e34a54d44c0fdafc2ef" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36085 } }
I20260806 19:48:04.606411 12148 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:48:04.606566 12148 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37db17e6e91b4e34a54d44c0fdafc2ef, State: Initialized, Role: FOLLOWER
I20260806 19:48:04.607179 12148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37db17e6e91b4e34a54d44c0fdafc2ef" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36085 } }
I20260806 19:48:04.607561 12148 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:48:04.607755 12148 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:48:04.608024 12148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:48:04.611491 12148 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37db17e6e91b4e34a54d44c0fdafc2ef" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36085 } }
I20260806 19:48:04.612152 12148 leader_election.cc:302] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37db17e6e91b4e34a54d44c0fdafc2ef; no voters:
I20260806 19:48:04.612974 12148 leader_election.cc:288] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:48:04.613169 12159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:48:04.615471 12148 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:48:04.616622 12151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:04.607690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47501 (local address 127.21.202.126:36085)
0806 19:48:04.607904 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:04.607910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:04.608369 (+ 459us) server_negotiation.cc:407] Connection header received
0806 19:48:04.608652 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:04.608656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:04.608728 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:04.608974 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:04.609612 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:04.610512 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:04.612666 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:04.613522 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:04.614567 (+ 1045us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:04.615410 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:04.615419 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:04.615427 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:48:04.615461 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:04.615522 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:48:04.615549 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:04.615781 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:04.616046 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 19:48:04.616122 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 19:48:04.620038 12159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [term 1 LEADER]: Becoming Leader. State: Replica: 37db17e6e91b4e34a54d44c0fdafc2ef, State: Running, Role: LEADER
I20260806 19:48:04.620828 12159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37db17e6e91b4e34a54d44c0fdafc2ef" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36085 } }
I20260806 19:48:04.624538 12161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37db17e6e91b4e34a54d44c0fdafc2ef. Latest consensus state: current_term: 1 leader_uuid: "37db17e6e91b4e34a54d44c0fdafc2ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37db17e6e91b4e34a54d44c0fdafc2ef" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36085 } } }
I20260806 19:48:04.625027 12161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:04.625221 12160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37db17e6e91b4e34a54d44c0fdafc2ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37db17e6e91b4e34a54d44c0fdafc2ef" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36085 } } }
I20260806 19:48:04.625530 12160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef [sys.catalog]: This master's current role is: LEADER
W20260806 19:48:04.631294 12173 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:48:04.631423 12173 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:48:04.631584 12174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:48:04.633438 12174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:48:04.642023 12174 catalog_manager.cc:1347] Generated new cluster ID: 5cbf84c998954c26a4d04d7f9db80516
I20260806 19:48:04.642159 12174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:48:04.666433 12174 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:48:04.668010 12174 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:48:04.678278 12174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37db17e6e91b4e34a54d44c0fdafc2ef: Generated new TSK 0
I20260806 19:48:04.679237 12174 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:48:04.685504 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 12081
W20260806 19:48:04.686807 12143 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52671
W20260806 19:48:04.687723 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 12081: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:48:04.687897 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 12081
W20260806 19:48:04.708685 12066 connection.cc:535] client connection to 127.21.202.126:36085 recv error: Network error: failed to read from TLS socket (remote: 127.21.202.126:36085): Connection reset by peer (error 104)
2026-08-06T19:48:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2108 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:48:04 dist-test-slave-77v9 krb5kdc[12180](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:48:04 dist-test-slave-77v9 krb5kdc[12180](info): set up 2 sockets
Aug 06 19:48:04 dist-test-slave-77v9 krb5kdc[12180](info): commencing operation
krb5kdc: starting...
W20260806 19:48:05.877391 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:48:06 dist-test-slave-77v9 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:48:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:48:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:48:06.226249 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:36251
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:36251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:48:06.591144 12193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:48:06.591558 12193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:48:06.591727 12193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:48:06.599443 12193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:48:06.599622 12193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:48:06.599735 12193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:48:06.599851 12193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:48:06.599936 12193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:48:06.610409 12193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:36251
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:36251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:48:06.611900 12193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:48:06.613332 12193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:48:06.626398 12199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:06.626470 12201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:06.626492 12198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:06.627421 12193 server_base.cc:1034] running on GCE node
I20260806 19:48:06.628463 12193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:48:06.630394 12193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:48:06.631637 12193 hybrid_clock.cc:648] HybridClock initialized: now 1786045686631598 us; error 37 us; skew 500 ppm
I20260806 19:48:06.632182 12193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:48:06 dist-test-slave-77v9 krb5kdc[12180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045686, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:48:06.642114 12193 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:48:06.644228 12193 webserver.cc:460] Webserver started at http://127.21.202.126:35651/ using document root <none> and password file <none>
I20260806 19:48:06.645136 12193 fs_manager.cc:359] Metadata directory not provided
I20260806 19:48:06.645320 12193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:48:06.645676 12193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:48:06.649799 12193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "504a8563afb746159ac4a0949061dc20"
format_stamp: "Formatted at 2026-08-06 19:48:06 on dist-test-slave-77v9"
server_key: "39b76a5cc44540a9a1f264f703b97942"
server_key_iv: "a74e0b3200a518505c1a81e52b6ae993"
server_key_version: "encryptionkey@0"
I20260806 19:48:06.650852 12193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "504a8563afb746159ac4a0949061dc20"
format_stamp: "Formatted at 2026-08-06 19:48:06 on dist-test-slave-77v9"
server_key: "39b76a5cc44540a9a1f264f703b97942"
server_key_iv: "a74e0b3200a518505c1a81e52b6ae993"
server_key_version: "encryptionkey@0"
I20260806 19:48:06.656771 12193 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 19:48:06.660768 12208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:48:06.662499 12193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:48:06.662741 12193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "504a8563afb746159ac4a0949061dc20"
format_stamp: "Formatted at 2026-08-06 19:48:06 on dist-test-slave-77v9"
server_key: "39b76a5cc44540a9a1f264f703b97942"
server_key_iv: "a74e0b3200a518505c1a81e52b6ae993"
server_key_version: "encryptionkey@0"
I20260806 19:48:06.662992 12193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:48:06.669674 12193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:48:06.674631 12193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:48:06.674996 12193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:48:06.696938 12193 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:36251
I20260806 19:48:06.696957 12259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:36251 every 8 connection(s)
I20260806 19:48:06.698778 12193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:48:06.700903 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 12193
I20260806 19:48:06.701136 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:48:06.701658 22313 external_mini_cluster.cc:1428] Setting key 139d4076ee6f6a838bd84edd29935368
I20260806 19:48:06.703689 12260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:48:06.710508 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:06.703576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50081 (local address 127.21.202.126:36251)
0806 19:48:06.704540 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:06.704553 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:06.704598 (+ 45us) server_negotiation.cc:407] Connection header received
0806 19:48:06.706467 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:06.706491 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:06.706870 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:06.707257 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:06.708620 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:06.708627 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:06.708640 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:48:06.708657 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:06.708743 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:48:06.708784 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:06.709075 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:06.709238 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 19:48:06.709479 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":480,"threads_started":1}
I20260806 19:48:06.715164 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:48:06.718111 12260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20: Bootstrap starting.
I20260806 19:48:06.722512 12260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20: Neither blocks nor log segments found. Creating new log.
I20260806 19:48:06.723068 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:06.720037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41553 (local address 127.21.202.126:36251)
0806 19:48:06.720292 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:06.720298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:06.720706 (+ 408us) server_negotiation.cc:407] Connection header received
0806 19:48:06.720937 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:06.720942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:06.720996 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:06.721087 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:06.722185 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:06.722190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:06.722193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:48:06.722214 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:06.722263 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:48:06.722282 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:06.722403 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:06.722598 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 19:48:06.722700 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 19:48:06.723960 12260 log.cc:824] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20: Log is configured to *not* fsync() on all Append() calls
I20260806 19:48:06.727174 12260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20: No bootstrap required, opened a new log
I20260806 19:48:06.735066 12260 raft_consensus.cc:348] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "504a8563afb746159ac4a0949061dc20" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36251 } }
I20260806 19:48:06.735493 12260 raft_consensus.cc:374] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:48:06.735602 12260 raft_consensus.cc:729] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 504a8563afb746159ac4a0949061dc20, State: Initialized, Role: FOLLOWER
I20260806 19:48:06.736202 12260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "504a8563afb746159ac4a0949061dc20" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36251 } }
I20260806 19:48:06.736423 12260 raft_consensus.cc:388] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:48:06.736584 12260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:48:06.736766 12260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:48:06.739446 12260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "504a8563afb746159ac4a0949061dc20" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36251 } }
I20260806 19:48:06.739980 12260 leader_election.cc:302] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 504a8563afb746159ac4a0949061dc20; no voters:
I20260806 19:48:06.740618 12260 leader_election.cc:288] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:48:06.740859 12270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:48:06.741662 12270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [term 1 LEADER]: Becoming Leader. State: Replica: 504a8563afb746159ac4a0949061dc20, State: Running, Role: LEADER
I20260806 19:48:06.742264 12270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "504a8563afb746159ac4a0949061dc20" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36251 } }
I20260806 19:48:06.742938 12260 sys_catalog.cc:564] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:48:06.746608 12271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "504a8563afb746159ac4a0949061dc20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "504a8563afb746159ac4a0949061dc20" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36251 } } }
I20260806 19:48:06.747016 12271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:06.746397 12272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 504a8563afb746159ac4a0949061dc20. Latest consensus state: current_term: 1 leader_uuid: "504a8563afb746159ac4a0949061dc20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "504a8563afb746159ac4a0949061dc20" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 36251 } } }
I20260806 19:48:06.747614 12272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20 [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:06.749507 12280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:48:06.754015 12280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:48:06.763003 12280 catalog_manager.cc:1347] Generated new cluster ID: 6a9a526d27874708b9fcb3d8d39812e4
I20260806 19:48:06.763278 12280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:48:06.776080 12280 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:48:06.776623 12280 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:48:06.787214 12280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 504a8563afb746159ac4a0949061dc20: Generated new TSK 0
I20260806 19:48:06.788378 12280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:48:06.802521 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 12193
W20260806 19:48:06.804157 12255 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50081
W20260806 19:48:06.804939 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 12193: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:48:06.805088 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 12193
2026-08-06T19:48:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2115 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:48:06 dist-test-slave-77v9 krb5kdc[12291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:48:06 dist-test-slave-77v9 krb5kdc[12291](info): set up 2 sockets
Aug 06 19:48:06 dist-test-slave-77v9 krb5kdc[12291](info): commencing operation
krb5kdc: starting...
W20260806 19:48:07.965416 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:48:08 dist-test-slave-77v9 krb5kdc[12291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:48:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:48:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:48:08.305421 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:41339
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:41339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:48:08.689380 12305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:48:08.689693 12305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:48:08.689841 12305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:48:08.697500 12305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:48:08.697690 12305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:48:08.697809 12305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:48:08.697924 12305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:48:08.698002 12305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:48:08.708779 12305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:41339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:41339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:48:08.710276 12305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:48:08.711633 12305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:48:08.724131 12313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:08.724254 12311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:08.724159 12310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:08.725523 12305 server_base.cc:1034] running on GCE node
I20260806 19:48:08.726367 12305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:48:08.727972 12305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:48:08.729223 12305 hybrid_clock.cc:648] HybridClock initialized: now 1786045688729156 us; error 58 us; skew 500 ppm
I20260806 19:48:08.729609 12305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:48:08 dist-test-slave-77v9 krb5kdc[12291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045688, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:48:08.732903 12305 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:48:08.734534 12305 webserver.cc:460] Webserver started at http://127.21.202.126:33611/ using document root <none> and password file <none>
I20260806 19:48:08.735276 12305 fs_manager.cc:359] Metadata directory not provided
I20260806 19:48:08.735451 12305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:48:08.735782 12305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:48:08.738946 12305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "33d464ab2cc04de2b8ccdb1e6b3c667d"
format_stamp: "Formatted at 2026-08-06 19:48:08 on dist-test-slave-77v9"
server_key: "71f2db97f1c61e2285776d8910313832"
server_key_iv: "3636f8cc4f94943f1e9d8868268b472a"
server_key_version: "encryptionkey@0"
I20260806 19:48:08.739756 12305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "33d464ab2cc04de2b8ccdb1e6b3c667d"
format_stamp: "Formatted at 2026-08-06 19:48:08 on dist-test-slave-77v9"
server_key: "71f2db97f1c61e2285776d8910313832"
server_key_iv: "3636f8cc4f94943f1e9d8868268b472a"
server_key_version: "encryptionkey@0"
I20260806 19:48:08.745286 12305 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 19:48:08.749523 12320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:48:08.751039 12305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:48:08.751241 12305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "33d464ab2cc04de2b8ccdb1e6b3c667d"
format_stamp: "Formatted at 2026-08-06 19:48:08 on dist-test-slave-77v9"
server_key: "71f2db97f1c61e2285776d8910313832"
server_key_iv: "3636f8cc4f94943f1e9d8868268b472a"
server_key_version: "encryptionkey@0"
I20260806 19:48:08.751464 12305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:48:08.768085 12305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:48:08.772936 12305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:48:08.773252 12305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:48:08.794250 12305 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:41339
I20260806 19:48:08.794283 12371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:41339 every 8 connection(s)
I20260806 19:48:08.795718 12305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:48:08.799971 12372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:48:08.800071 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 12305
I20260806 19:48:08.800256 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:48:08.800916 22313 external_mini_cluster.cc:1428] Setting key 5bd8f1bddbec3408af5d47a33a1b1218
I20260806 19:48:08.816249 12372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d: Bootstrap starting.
Aug 06 19:48:08 dist-test-slave-77v9 krb5kdc[12291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:48:08.819561 12372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d: Neither blocks nor log segments found. Creating new log.
I20260806 19:48:08.820664 12372 log.cc:824] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d: Log is configured to *not* fsync() on all Append() calls
I20260806 19:48:08.823560 12372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d: No bootstrap required, opened a new log
I20260806 19:48:08.825438 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:08.802886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44877 (local address 127.21.202.126:41339)
0806 19:48:08.803777 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:08.803789 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:08.803835 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:48:08.805586 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:08.805606 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:08.805997 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:08.806336 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:08.808551 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:08.810015 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:08.811884 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:08.812634 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:08.812905 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:08.817740 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:08.817748 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:08.817792 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:08.817835 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:08.821082 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:08.821852 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:08.821859 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:08.821865 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:08.822032 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:08.822465 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:08.822471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:08.822474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:08.823026 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:08.823323 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:08.823803 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 19:48:08.824285 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":422,"threads_started":1}
I20260806 19:48:08.829384 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:48:08.831911 12372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d464ab2cc04de2b8ccdb1e6b3c667d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41339 } }
I20260806 19:48:08.832295 12372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:48:08.832420 12372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d464ab2cc04de2b8ccdb1e6b3c667d, State: Initialized, Role: FOLLOWER
I20260806 19:48:08.833092 12372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d464ab2cc04de2b8ccdb1e6b3c667d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41339 } }
I20260806 19:48:08.833352 12372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:48:08.833498 12372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:48:08.833683 12372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:48:08.836048 12372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d464ab2cc04de2b8ccdb1e6b3c667d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41339 } }
I20260806 19:48:08.836571 12372 leader_election.cc:302] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33d464ab2cc04de2b8ccdb1e6b3c667d; no voters:
I20260806 19:48:08.837224 12372 leader_election.cc:288] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:48:08.837399 12382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:48:08.838620 12382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [term 1 LEADER]: Becoming Leader. State: Replica: 33d464ab2cc04de2b8ccdb1e6b3c667d, State: Running, Role: LEADER
I20260806 19:48:08.839449 12372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:48:08.839310 12382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d464ab2cc04de2b8ccdb1e6b3c667d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41339 } }
I20260806 19:48:08.847718 12383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33d464ab2cc04de2b8ccdb1e6b3c667d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d464ab2cc04de2b8ccdb1e6b3c667d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41339 } } }
I20260806 19:48:08.848210 12383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:08.847739 12384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33d464ab2cc04de2b8ccdb1e6b3c667d. Latest consensus state: current_term: 1 leader_uuid: "33d464ab2cc04de2b8ccdb1e6b3c667d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d464ab2cc04de2b8ccdb1e6b3c667d" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 41339 } } }
I20260806 19:48:08.849049 12384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:08.849356 12397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:48:08.855425 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:08.834058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43925 (local address 127.21.202.126:41339)
0806 19:48:08.834321 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:08.834330 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:08.834871 (+ 541us) server_negotiation.cc:407] Connection header received
0806 19:48:08.835110 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:08.835114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:08.835186 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:08.835292 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:08.836910 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:08.838240 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:08.840080 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:08.840894 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:08.841114 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:08.845016 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:08.845029 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:08.845033 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:08.845067 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:08.849874 (+ 4807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:08.850969 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:08.850975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:08.850978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:08.851147 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:08.851795 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:08.851804 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:08.851808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:08.852701 (+ 893us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:08.854410 (+ 1709us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:08.854590 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 19:48:08.854908 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 19:48:08.856868 12397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:48:08.866947 12397 catalog_manager.cc:1347] Generated new cluster ID: cfd590cefbd847db83acf69626eee3be
I20260806 19:48:08.867084 12397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:48:08.890226 12397 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:48:08.891644 12397 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:48:08.902421 12397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33d464ab2cc04de2b8ccdb1e6b3c667d: Generated new TSK 0
I20260806 19:48:08.903272 12397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:48:08.917752 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 12305
I20260806 19:48:08.919256 12367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:48:09.017341 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 12305
2026-08-06T19:48:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2211 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:48:09 dist-test-slave-77v9 krb5kdc[12404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:48:09 dist-test-slave-77v9 krb5kdc[12404](info): set up 2 sockets
Aug 06 19:48:09 dist-test-slave-77v9 krb5kdc[12404](info): commencing operation
krb5kdc: starting...
W20260806 19:48:10.186674 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.007s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:48:10 dist-test-slave-77v9 krb5kdc[12404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:48:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:48:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:48:10.532635 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:40903
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:38931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:40903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:48:10.902911 12417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:48:10.903244 12417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:48:10.903349 12417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:48:10.911860 12417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:48:10.911994 12417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:48:10.912047 12417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:48:10.912114 12417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:48:10.912171 12417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:48:10.923168 12417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:38931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:40903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:40903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:48:10.924515 12417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:48:10.925796 12417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:48:10.938102 12422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:10.938223 12425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:10.938169 12423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:10.938974 12417 server_base.cc:1034] running on GCE node
I20260806 19:48:10.939885 12417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:48:10.941597 12417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:48:10.942849 12417 hybrid_clock.cc:648] HybridClock initialized: now 1786045690942802 us; error 34 us; skew 500 ppm
I20260806 19:48:10.943249 12417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:48:10 dist-test-slave-77v9 krb5kdc[12404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:48:10.947033 12417 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:48:10.948863 12417 webserver.cc:460] Webserver started at http://127.21.202.126:39457/ using document root <none> and password file <none>
I20260806 19:48:10.949671 12417 fs_manager.cc:359] Metadata directory not provided
I20260806 19:48:10.949832 12417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:48:10.950163 12417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:48:10.953460 12417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "eb5ae2d536cd481ab8e5d066303a4cc5"
format_stamp: "Formatted at 2026-08-06 19:48:10 on dist-test-slave-77v9"
server_key: "f220a2a2b6f41ff034b28f9433e0e1d0"
server_key_iv: "135fe9c6e411589a98ca96e7d5410de3"
server_key_version: "encryptionkey@0"
I20260806 19:48:10.954301 12417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "eb5ae2d536cd481ab8e5d066303a4cc5"
format_stamp: "Formatted at 2026-08-06 19:48:10 on dist-test-slave-77v9"
server_key: "f220a2a2b6f41ff034b28f9433e0e1d0"
server_key_iv: "135fe9c6e411589a98ca96e7d5410de3"
server_key_version: "encryptionkey@0"
I20260806 19:48:10.960325 12417 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:48:10.964272 12432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:48:10.965845 12417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:48:10.966110 12417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "eb5ae2d536cd481ab8e5d066303a4cc5"
format_stamp: "Formatted at 2026-08-06 19:48:10 on dist-test-slave-77v9"
server_key: "f220a2a2b6f41ff034b28f9433e0e1d0"
server_key_iv: "135fe9c6e411589a98ca96e7d5410de3"
server_key_version: "encryptionkey@0"
I20260806 19:48:10.966305 12417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:48:10.984108 12417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:48:10.989064 12417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:48:10.989396 12417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:48:11.010609 12417 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:40903
I20260806 19:48:11.010618 12484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:40903 every 8 connection(s)
I20260806 19:48:11.012082 12417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:48:11.015981 12485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:48:11.017302 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 12417
I20260806 19:48:11.017489 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:48:11.017954 22313 external_mini_cluster.cc:1428] Setting key d80a88889cde35da1e98a5be19cacbfa
I20260806 19:48:11.031009 12485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5: Bootstrap starting.
I20260806 19:48:11.033739 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:11.020120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53509 (local address 127.21.202.126:40903)
0806 19:48:11.021051 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:11.021071 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:11.021122 (+ 51us) server_negotiation.cc:407] Connection header received
0806 19:48:11.022861 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:11.022894 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:11.023456 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:11.023925 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:11.024486 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:11.026424 (+ 1938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:11.027981 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:11.029115 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:11.029303 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:11.030658 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:11.030670 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:11.030689 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:48:11.031452 (+ 763us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:11.031562 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:48:11.031602 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:11.031711 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:11.031993 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 19:48:11.032291 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":408,"threads_started":1}
I20260806 19:48:11.035393 12485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5: Neither blocks nor log segments found. Creating new log.
I20260806 19:48:11.036612 12485 log.cc:824] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5: Log is configured to *not* fsync() on all Append() calls
I20260806 19:48:11.037911 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:48:11.040352 12485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5: No bootstrap required, opened a new log
I20260806 19:48:11.052173 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:11.044220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56099 (local address 127.21.202.126:40903)
0806 19:48:11.044447 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:11.044454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:11.044949 (+ 495us) server_negotiation.cc:407] Connection header received
0806 19:48:11.045185 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:11.045191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:11.045288 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:11.045426 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:11.045974 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:11.047273 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:11.049133 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:11.049954 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:11.050111 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:11.051231 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:11.051250 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:11.051255 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:48:11.051280 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:11.051330 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:48:11.051354 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:11.051465 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:11.051640 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 19:48:11.051737 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 19:48:11.051903 12485 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5ae2d536cd481ab8e5d066303a4cc5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40903 } }
I20260806 19:48:11.052470 12485 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:48:11.052632 12485 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5ae2d536cd481ab8e5d066303a4cc5, State: Initialized, Role: FOLLOWER
I20260806 19:48:11.053419 12485 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5ae2d536cd481ab8e5d066303a4cc5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40903 } }
I20260806 19:48:11.053748 12485 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:48:11.053921 12485 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:48:11.054132 12485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:48:11.056638 12485 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5ae2d536cd481ab8e5d066303a4cc5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40903 } }
I20260806 19:48:11.057252 12485 leader_election.cc:302] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb5ae2d536cd481ab8e5d066303a4cc5; no voters:
I20260806 19:48:11.057888 12485 leader_election.cc:288] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:48:11.058028 12495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:48:11.058796 12495 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [term 1 LEADER]: Becoming Leader. State: Replica: eb5ae2d536cd481ab8e5d066303a4cc5, State: Running, Role: LEADER
I20260806 19:48:11.059499 12495 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5ae2d536cd481ab8e5d066303a4cc5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40903 } }
I20260806 19:48:11.060318 12485 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:48:11.062918 12497 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb5ae2d536cd481ab8e5d066303a4cc5. Latest consensus state: current_term: 1 leader_uuid: "eb5ae2d536cd481ab8e5d066303a4cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5ae2d536cd481ab8e5d066303a4cc5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40903 } } }
I20260806 19:48:11.063336 12497 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:11.065111 12496 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb5ae2d536cd481ab8e5d066303a4cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5ae2d536cd481ab8e5d066303a4cc5" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 40903 } } }
I20260806 19:48:11.065519 12496 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5 [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:11.066962 12504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:48:11.077965 12504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:48:11.087613 12504 catalog_manager.cc:1347] Generated new cluster ID: 488d7f4b1fa64040a3a01964a937115d
I20260806 19:48:11.087769 12504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:48:11.103073 12504 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:48:11.105077 12504 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:48:11.113404 12504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb5ae2d536cd481ab8e5d066303a4cc5: Generated new TSK 0
I20260806 19:48:11.114499 12504 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:48:11.122743 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 12417
W20260806 19:48:11.123885 12480 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53509
W20260806 19:48:11.124568 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 12417: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:48:11.124712 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 12417
2026-08-06T19:48:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2104 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:48:11 dist-test-slave-77v9 krb5kdc[12516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:48:11 dist-test-slave-77v9 krb5kdc[12516](info): set up 2 sockets
Aug 06 19:48:11 dist-test-slave-77v9 krb5kdc[12516](info): commencing operation
krb5kdc: starting...
W20260806 19:48:12.289494 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:48:12 dist-test-slave-77v9 krb5kdc[12516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:48:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:48:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:48:12.624136 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:37251
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:37251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:48:12.973269 12529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:48:12.973639 12529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:48:12.973795 12529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:48:12.981232 12529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:48:12.981401 12529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:48:12.981503 12529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:48:12.981599 12529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:48:12.981680 12529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:48:12.991902 12529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:37251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:37251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:48:12.993393 12529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:48:12.994745 12529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:48:13.007323 12537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:13.007349 12535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:13.007668 12529 server_base.cc:1034] running on GCE node
W20260806 19:48:13.007249 12534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:13.008697 12529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:48:13.010206 12529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:48:13.011461 12529 hybrid_clock.cc:648] HybridClock initialized: now 1786045693011424 us; error 35 us; skew 500 ppm
I20260806 19:48:13.011847 12529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:48:13 dist-test-slave-77v9 krb5kdc[12516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045693, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:48:13.019265 12529 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:48:13.020921 12529 webserver.cc:460] Webserver started at http://127.21.202.126:33889/ using document root <none> and password file <none>
I20260806 19:48:13.021651 12529 fs_manager.cc:359] Metadata directory not provided
I20260806 19:48:13.021802 12529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:48:13.022121 12529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:48:13.025311 12529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "49d139acbfc746bf8cd2e5db05237cd0"
format_stamp: "Formatted at 2026-08-06 19:48:13 on dist-test-slave-77v9"
server_key: "7091d89f8f7bbc3ca752c75bef77e142"
server_key_iv: "53cdb8520ade8f987f83969d8a177b09"
server_key_version: "encryptionkey@0"
I20260806 19:48:13.026121 12529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "49d139acbfc746bf8cd2e5db05237cd0"
format_stamp: "Formatted at 2026-08-06 19:48:13 on dist-test-slave-77v9"
server_key: "7091d89f8f7bbc3ca752c75bef77e142"
server_key_iv: "53cdb8520ade8f987f83969d8a177b09"
server_key_version: "encryptionkey@0"
I20260806 19:48:13.031579 12529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:48:13.035303 12544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:48:13.036979 12529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 19:48:13.037175 12529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "49d139acbfc746bf8cd2e5db05237cd0"
format_stamp: "Formatted at 2026-08-06 19:48:13 on dist-test-slave-77v9"
server_key: "7091d89f8f7bbc3ca752c75bef77e142"
server_key_iv: "53cdb8520ade8f987f83969d8a177b09"
server_key_version: "encryptionkey@0"
I20260806 19:48:13.037393 12529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:48:13.043957 12529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:48:13.048532 12529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:48:13.048929 12529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:48:13.068046 12529 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:37251
I20260806 19:48:13.068086 12595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:37251 every 8 connection(s)
I20260806 19:48:13.069566 12529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:48:13.074035 12596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:48:13.077256 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 12529
I20260806 19:48:13.077446 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:48:13.077880 22313 external_mini_cluster.cc:1428] Setting key 5abbf2b5a55196168d78ed71c55dcb68
I20260806 19:48:13.087620 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:13.079886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35273 (local address 127.21.202.126:37251)
0806 19:48:13.080982 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:13.081007 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:13.081075 (+ 68us) server_negotiation.cc:407] Connection header received
0806 19:48:13.083526 (+ 2451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:13.083556 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:13.084001 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:13.084473 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:13.085527 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:13.085536 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:13.085554 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:48:13.085583 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:13.085648 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:48:13.085677 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:13.085920 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:13.086062 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 19:48:13.086265 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":772,"thread_start_us":502,"threads_started":1}
I20260806 19:48:13.089236 12596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0: Bootstrap starting.
I20260806 19:48:13.091508 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:48:13.092659 12596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0: Neither blocks nor log segments found. Creating new log.
I20260806 19:48:13.094009 12596 log.cc:824] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0: Log is configured to *not* fsync() on all Append() calls
I20260806 19:48:13.097561 12596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0: No bootstrap required, opened a new log
I20260806 19:48:13.101038 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:13.097567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43151 (local address 127.21.202.126:37251)
0806 19:48:13.097826 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:13.097836 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:13.098830 (+ 994us) server_negotiation.cc:407] Connection header received
0806 19:48:13.099142 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:13.099148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:13.099209 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:13.099324 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:13.100334 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:13.100340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:13.100345 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:48:13.100364 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:13.100421 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:48:13.100441 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:13.100579 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:13.100666 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 19:48:13.100742 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 19:48:13.107237 12596 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d139acbfc746bf8cd2e5db05237cd0" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37251 } }
I20260806 19:48:13.107642 12596 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:48:13.107748 12596 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d139acbfc746bf8cd2e5db05237cd0, State: Initialized, Role: FOLLOWER
I20260806 19:48:13.108407 12596 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d139acbfc746bf8cd2e5db05237cd0" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37251 } }
I20260806 19:48:13.108706 12596 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:48:13.108896 12596 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:48:13.109097 12596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:48:13.111318 12596 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d139acbfc746bf8cd2e5db05237cd0" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37251 } }
I20260806 19:48:13.111824 12596 leader_election.cc:302] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49d139acbfc746bf8cd2e5db05237cd0; no voters:
I20260806 19:48:13.112475 12596 leader_election.cc:288] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:48:13.112660 12606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:48:13.113487 12606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [term 1 LEADER]: Becoming Leader. State: Replica: 49d139acbfc746bf8cd2e5db05237cd0, State: Running, Role: LEADER
I20260806 19:48:13.114135 12606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d139acbfc746bf8cd2e5db05237cd0" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37251 } }
I20260806 19:48:13.114674 12596 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:48:13.117604 12608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49d139acbfc746bf8cd2e5db05237cd0. Latest consensus state: current_term: 1 leader_uuid: "49d139acbfc746bf8cd2e5db05237cd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d139acbfc746bf8cd2e5db05237cd0" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37251 } } }
I20260806 19:48:13.118225 12608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:13.118170 12607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49d139acbfc746bf8cd2e5db05237cd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d139acbfc746bf8cd2e5db05237cd0" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 37251 } } }
I20260806 19:48:13.118628 12607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0 [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:13.119510 12615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:48:13.125608 12615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:48:13.134331 12615 catalog_manager.cc:1347] Generated new cluster ID: c0764218dada48b7bd187198f3b433ec
I20260806 19:48:13.134488 12615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:48:13.152544 12615 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:48:13.153254 12615 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:48:13.165349 12615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49d139acbfc746bf8cd2e5db05237cd0: Generated new TSK 0
I20260806 19:48:13.166553 12615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:48:13.180397 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 12529
W20260806 19:48:13.181748 12591 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35273
W20260806 19:48:13.182497 22313 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 12529: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:48:13.182646 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 12529
2026-08-06T19:48:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2059 ms)
[----------] 18 tests from AuthTokenIssuingTest (38916 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:48:13 dist-test-slave-77v9 krb5kdc[12627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:48:13 dist-test-slave-77v9 krb5kdc[12627](info): set up 2 sockets
Aug 06 19:48:13 dist-test-slave-77v9 krb5kdc[12627](info): commencing operation
krb5kdc: starting...
W20260806 19:48:14.373829 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:48:14 dist-test-slave-77v9 krb5kdc[12627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:48:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:48:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:48:14.718006 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:39333
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:39333,127.21.202.125:39149,127.21.202.124:37927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:48:15.084575 12640 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 19:48:15.085007 12640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:48:15.085152 12640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:48:15.085314 12640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:48:15.093173 12640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:48:15.093281 12640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:48:15.093370 12640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:48:15.093449 12640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:48:15.093552 12640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:48:15.103932 12640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:39333,127.21.202.125:39149,127.21.202.124:37927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:39333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:48:15.105902 12640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:48:15.107700 12640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:48:15.120398 12646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:15.120582 12645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:15.120703 12648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:15.121340 12640 server_base.cc:1034] running on GCE node
I20260806 19:48:15.122131 12640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:48:15.123669 12640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:48:15.124989 12640 hybrid_clock.cc:648] HybridClock initialized: now 1786045695124871 us; error 61 us; skew 500 ppm
I20260806 19:48:15.125339 12640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:48:15 dist-test-slave-77v9 krb5kdc[12627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045695, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:48:15.128979 12640 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:48:15.130662 12640 webserver.cc:460] Webserver started at http://127.21.202.126:40435/ using document root <none> and password file <none>
I20260806 19:48:15.131433 12640 fs_manager.cc:359] Metadata directory not provided
I20260806 19:48:15.131529 12640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:48:15.131799 12640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:48:15.135020 12640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "eba2682bfd684ad582ea647149b351eb"
format_stamp: "Formatted at 2026-08-06 19:48:15 on dist-test-slave-77v9"
server_key: "408fe93d0eacd2b233df19a24d6039aa"
server_key_iv: "0fb56af12450bfe09a778d05e5df8dcd"
server_key_version: "encryptionkey@0"
I20260806 19:48:15.135823 12640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "eba2682bfd684ad582ea647149b351eb"
format_stamp: "Formatted at 2026-08-06 19:48:15 on dist-test-slave-77v9"
server_key: "408fe93d0eacd2b233df19a24d6039aa"
server_key_iv: "0fb56af12450bfe09a778d05e5df8dcd"
server_key_version: "encryptionkey@0"
I20260806 19:48:15.141558 12640 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 19:48:15.145565 12655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:48:15.147147 12640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:48:15.147367 12640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "eba2682bfd684ad582ea647149b351eb"
format_stamp: "Formatted at 2026-08-06 19:48:15 on dist-test-slave-77v9"
server_key: "408fe93d0eacd2b233df19a24d6039aa"
server_key_iv: "0fb56af12450bfe09a778d05e5df8dcd"
server_key_version: "encryptionkey@0"
I20260806 19:48:15.147619 12640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:48:15.180439 12640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:48:15.185559 12640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:48:15.185966 12640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:48:15.207037 12640 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:39333
I20260806 19:48:15.207054 12706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:39333 every 8 connection(s)
I20260806 19:48:15.208732 12640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:48:15.212754 12707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:48:15.214907 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 12640
I20260806 19:48:15.215133 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:48:15.215794 22313 external_mini_cluster.cc:1428] Setting key 6aa5c3172486f89819f53388674a1380
I20260806 19:48:15.219875 12707 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:48:15 dist-test-slave-77v9 krb5kdc[12627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045695, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.202.125@KRBTEST.COM; defaulting to no policy
I20260806 19:48:15.248382 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.223958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55614 (local address 127.21.202.126:39333)
0806 19:48:15.225877 (+ 1919us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:15.225888 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:15.225957 (+ 69us) server_negotiation.cc:407] Connection header received
0806 19:48:15.227564 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:15.227577 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:15.227931 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:15.228175 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:15.229804 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.231180 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.232740 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.233458 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.233696 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:15.239194 (+ 5498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:15.239205 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:15.239221 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:15.239256 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:15.243627 (+ 4371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:15.244415 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:15.244422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:15.244424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:15.244576 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:15.245057 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:15.245066 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:15.245068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:15.245612 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:15.245834 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:15.246384 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 19:48:15.246808 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":1658,"spinlock_wait_cycles":54656,"thread_start_us":1423,"threads_started":1}
I20260806 19:48:15.248409 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.223959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:39333 (local address 127.0.0.1:55614)
0806 19:48:15.225044 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.225108 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:15.227142 (+ 2034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:15.228285 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:15.228291 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:15.229046 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:15.229556 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:15.229579 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.231371 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.231377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:15.232578 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:15.232591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.233732 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.233740 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:15.233939 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:15.234691 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:15.234717 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:15.238875 (+ 4158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:15.243791 (+ 4916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:15.243804 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:15.243818 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:15.244251 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:15.244699 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:15.244705 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:15.244708 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:15.244944 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:15.245750 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:15.245759 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:15.246000 (+ 241us) client_negotiation.cc:769] Sending connection context
0806 19:48:15.246248 (+ 248us) client_negotiation.cc:240] Negotiation successful
0806 19:48:15.246964 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":166,"thread_start_us":433,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.125@KRBTEST.COM" created.
I20260806 19:48:15.251871 12707 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:48:15.253481 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.253007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:39149 (local address 127.0.0.1:52070)
0806 19:48:15.253233 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.253356 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260806 19:48:15.253908 12658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
I20260806 19:48:15.255689 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.255219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:39149 (local address 127.0.0.1:52078)
0806 19:48:15.255424 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.255518 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260806 19:48:15.255968 12707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:39149: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.125.keytab.
Entry for principal kudu/127.21.202.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.125.keytab.
I20260806 19:48:15.288710 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.125:39149
--webserver_interface=127.21.202.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.125.keytab
--principal=kudu/127.21.202.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:39333,127.21.202.125:39149,127.21.202.124:37927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:48:15.305346 12707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } attempt: 1
I20260806 19:48:15.306689 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.306175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:39149 (local address 127.0.0.1:52090)
0806 19:48:15.306408 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.306546 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 19:48:15.307930 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.307500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:39149 (local address 127.0.0.1:52102)
0806 19:48:15.307698 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.307782 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260806 19:48:15.308144 12707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:39149: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
I20260806 19:48:15.376418 12707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } attempt: 2
I20260806 19:48:15.377808 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.377276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:39149 (local address 127.0.0.1:52116)
0806 19:48:15.377553 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.377653 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 19:48:15.379034 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.378628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:39149 (local address 127.0.0.1:52132)
0806 19:48:15.378801 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.378899 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 19:48:15.379223 12707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:39149: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
I20260806 19:48:15.470520 12707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } attempt: 3
I20260806 19:48:15.472025 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.471373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:39149 (local address 127.0.0.1:52134)
0806 19:48:15.471691 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.471843 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 19:48:15.473857 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.473236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:39149 (local address 127.0.0.1:52148)
0806 19:48:15.473460 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.473629 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260806 19:48:15.474243 12707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:39149: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
I20260806 19:48:15.617594 12707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } attempt: 4
I20260806 19:48:15.618978 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.618372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:39149 (local address 127.0.0.1:52160)
0806 19:48:15.618716 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.618827 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260806 19:48:15.620153 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.619725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:39149 (local address 127.0.0.1:52162)
0806 19:48:15.619903 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.619983 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 19:48:15.620396 12707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:39149: Network error: Client connection negotiation failed: client connection to 127.21.202.125:39149: connect: Connection refused (error 111)
W20260806 19:48:15.664467 12714 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 19:48:15.664856 12714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:48:15.664927 12714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:48:15.665016 12714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:48:15.672590 12714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:48:15.672713 12714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:48:15.672777 12714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:48:15.672901 12714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:48:15.672976 12714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:48:15.684033 12714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:39333,127.21.202.125:39149,127.21.202.124:37927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.125.keytab
--principal=kudu/127.21.202.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.125:39149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.202.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:48:15.685514 12714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:48:15.686993 12714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:48:15.700196 12722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:15.700242 12719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:15.700269 12720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:15.701249 12714 server_base.cc:1034] running on GCE node
I20260806 19:48:15.702171 12714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:48:15.703732 12714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:48:15.705024 12714 hybrid_clock.cc:648] HybridClock initialized: now 1786045695704927 us; error 56 us; skew 500 ppm
I20260806 19:48:15.705399 12714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:48:15 dist-test-slave-77v9 krb5kdc[12627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045695, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:48:15.709111 12714 init.cc:377] Logged in from keytab as kudu/127.21.202.125@KRBTEST.COM (short username kudu)
I20260806 19:48:15.710732 12714 webserver.cc:460] Webserver started at http://127.21.202.125:45437/ using document root <none> and password file <none>
I20260806 19:48:15.711481 12714 fs_manager.cc:359] Metadata directory not provided
I20260806 19:48:15.711575 12714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:48:15.711862 12714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:48:15.715104 12714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/data/instance:
uuid: "a18b062fa7bc429db9910366a751049d"
format_stamp: "Formatted at 2026-08-06 19:48:15 on dist-test-slave-77v9"
server_key: "e0b28bb3b929f63c2b6d4881661a1b57"
server_key_iv: "af0c783c8fc3adfa6332e1de77025f6c"
server_key_version: "encryptionkey@0"
I20260806 19:48:15.715919 12714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/wal/instance:
uuid: "a18b062fa7bc429db9910366a751049d"
format_stamp: "Formatted at 2026-08-06 19:48:15 on dist-test-slave-77v9"
server_key: "e0b28bb3b929f63c2b6d4881661a1b57"
server_key_iv: "af0c783c8fc3adfa6332e1de77025f6c"
server_key_version: "encryptionkey@0"
I20260806 19:48:15.721774 12714 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 19:48:15.725802 12729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:48:15.727469 12714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:48:15.727674 12714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/wal
uuid: "a18b062fa7bc429db9910366a751049d"
format_stamp: "Formatted at 2026-08-06 19:48:15 on dist-test-slave-77v9"
server_key: "e0b28bb3b929f63c2b6d4881661a1b57"
server_key_iv: "af0c783c8fc3adfa6332e1de77025f6c"
server_key_version: "encryptionkey@0"
I20260806 19:48:15.727928 12714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:48:15.744589 12714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:48:15.749312 12714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:48:15.749605 12714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:48:15.768569 12714 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.125:39149
I20260806 19:48:15.768622 12780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.125:39149 every 8 connection(s)
I20260806 19:48:15.770272 12714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/data/info.pb
I20260806 19:48:15.774214 12781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:48:15.774719 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 12714
I20260806 19:48:15.774909 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-1/wal/instance
I20260806 19:48:15.775394 22313 external_mini_cluster.cc:1428] Setting key ca98a1999303dc16014762ab4c30317d
I20260806 19:48:15.781844 12781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:48:15 dist-test-slave-77v9 krb5kdc[12627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045695, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.125@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:48:15.808208 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.784705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55624 (local address 127.21.202.126:39333)
0806 19:48:15.785313 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:15.785323 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:15.786310 (+ 987us) server_negotiation.cc:407] Connection header received
0806 19:48:15.788501 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:15.788506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:15.788580 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:15.788688 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:15.791073 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.791841 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.794852 (+ 3011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.795294 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.795415 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:15.801112 (+ 5697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:15.801119 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:15.801123 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:15.801191 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:15.804133 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:15.805417 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:15.805422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:15.805424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:15.805562 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:15.806092 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:15.806096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:15.806098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:15.806405 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:15.806543 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:15.807733 (+ 1190us) server_negotiation.cc:299] Negotiation successful
0806 19:48:15.807929 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":283,"threads_started":1}
I20260806 19:48:15.809988 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.785279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:39333 (local address 127.0.0.1:55624)
0806 19:48:15.786132 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.786180 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:15.788143 (+ 1963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:15.789036 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:15.789050 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:15.789938 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:15.790793 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:15.790822 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.792044 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.792054 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:15.794602 (+ 2548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:15.794632 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.795592 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.795607 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:15.796007 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:15.796946 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:15.796985 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:15.800717 (+ 3732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:15.804343 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:15.804358 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:15.804385 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:15.805219 (+ 834us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:15.805686 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:15.805694 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:15.805701 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:15.805973 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:15.806548 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:15.806561 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:15.807109 (+ 548us) client_negotiation.cc:769] Sending connection context
0806 19:48:15.807605 (+ 496us) client_negotiation.cc:240] Negotiation successful
0806 19:48:15.808366 (+ 761us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":362,"threads_started":1}
WARNING: no policy specified for kudu/127.21.202.124@KRBTEST.COM; defaulting to no policy
I20260806 19:48:15.812147 12781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.124@KRBTEST.COM" created.
Aug 06 19:48:15 dist-test-slave-77v9 krb5kdc[12627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045695, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.125@KRBTEST.COM for kudu/127.21.202.125@KRBTEST.COM
I20260806 19:48:15.830770 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.813028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:39149 (local address 127.0.0.1:52164)
0806 19:48:15.813192 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.813215 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:15.813369 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:15.814678 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:15.814686 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:15.815247 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:15.815482 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:15.815497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.816993 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.817002 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:15.818634 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:15.818653 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.819659 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.819665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:15.819883 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:15.820693 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:15.820712 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:15.823306 (+ 2594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:15.828199 (+ 4893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:15.828203 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:15.828206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:15.828669 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:15.829119 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:15.829127 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:15.829132 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:15.829342 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:15.830017 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:15.830021 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:15.830168 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 19:48:15.830281 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 19:48:15.830500 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 19:48:15.831357 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.813331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52164 (local address 127.21.202.125:39149)
0806 19:48:15.814136 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:15.814146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:15.814186 (+ 40us) server_negotiation.cc:407] Connection header received
0806 19:48:15.814246 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:15.814258 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:15.814485 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:15.814615 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:15.815689 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.816767 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.818816 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.819424 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.819661 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:15.823575 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:15.823596 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:15.823620 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:15.823666 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:15.828005 (+ 4339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:15.828851 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:15.828863 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:15.828866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:15.829005 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:15.829461 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:15.829464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:15.829466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:15.829871 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:15.830085 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:15.830461 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 19:48:15.830763 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":391,"threads_started":1}
I20260806 19:48:15.833868 12781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:48:15.835064 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.834469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49468)
0806 19:48:15.834673 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.834838 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 19:48:15.835384 12732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
I20260806 19:48:15.837281 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.836705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49484)
0806 19:48:15.836941 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.837126 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 19:48:15.837543 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:37927: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.124.keytab.
Entry for principal kudu/127.21.202.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.124.keytab.
I20260806 19:48:15.857122 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.124:37927
--webserver_interface=127.21.202.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.124.keytab
--principal=kudu/127.21.202.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:44979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:39333,127.21.202.125:39149,127.21.202.124:37927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:48:15.886947 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } attempt: 1
I20260806 19:48:15.888371 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.887790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49494)
0806 19:48:15.888033 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.888166 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 19:48:15.889501 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.889125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49506)
0806 19:48:15.889283 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.889378 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 19:48:15.889756 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:37927: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
I20260806 19:48:15.919641 12707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } attempt: 5
Aug 06 19:48:15 dist-test-slave-77v9 krb5kdc[12627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045695, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for kudu/127.21.202.125@KRBTEST.COM
I20260806 19:48:15.934273 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.920647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52174 (local address 127.21.202.125:39149)
0806 19:48:15.920883 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:15.920889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:15.920978 (+ 89us) server_negotiation.cc:407] Connection header received
0806 19:48:15.921209 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:15.921214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:15.921299 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:15.921421 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:15.922319 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.923092 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.924363 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.924854 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.925006 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:15.928468 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:15.928473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:15.928476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:15.928510 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:15.931369 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:15.932180 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:15.932186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:15.932190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:15.932321 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:15.932702 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:15.932708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:15.932711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:15.933126 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:15.933281 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:15.933684 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 19:48:15.933933 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 19:48:15.934269 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.920460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:39149 (local address 127.0.0.1:52174)
0806 19:48:15.920843 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.920879 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:15.921065 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:15.921448 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:15.921453 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:15.921956 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:15.922157 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:15.922167 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.923235 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.923241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:15.924194 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:15.924207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:15.925018 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:15.925025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:15.925262 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:15.925970 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:15.925989 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:15.928257 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:15.931552 (+ 3295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:15.931557 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:15.931562 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:15.932057 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:15.932443 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:15.932448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:15.932452 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:15.932593 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:15.933250 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:15.933255 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:15.933423 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 19:48:15.933571 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 19:48:15.933905 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 19:48:15.935384 12707 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:48:15.936594 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.936112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49516)
0806 19:48:15.936311 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.936412 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 19:48:15.937744 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.937276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49532)
0806 19:48:15.937458 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.937588 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 19:48:15.938022 12707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:37927: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
I20260806 19:48:15.958099 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } attempt: 2
I20260806 19:48:15.959400 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.958819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49538)
0806 19:48:15.959057 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.959174 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 19:48:15.960598 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.960180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49544)
0806 19:48:15.960353 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.960449 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 19:48:15.960852 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:37927: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
I20260806 19:48:15.989291 12707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } attempt: 1
I20260806 19:48:15.990613 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.990059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49556)
0806 19:48:15.990298 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.990411 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 19:48:15.991629 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:15.991268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49566)
0806 19:48:15.991418 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:15.991503 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 19:48:15.991842 12707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:37927: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
I20260806 19:48:16.052125 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } attempt: 3
I20260806 19:48:16.053891 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.053144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49574)
0806 19:48:16.053464 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.053624 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 19:48:16.055280 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.054644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49582)
0806 19:48:16.054946 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.055074 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
W20260806 19:48:16.055594 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:37927: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
I20260806 19:48:16.060072 12707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } attempt: 2
I20260806 19:48:16.061343 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.060729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49598)
0806 19:48:16.061009 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.061121 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 19:48:16.062631 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.062137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49604)
0806 19:48:16.062329 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.062441 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 19:48:16.063090 12707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:37927: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
I20260806 19:48:16.169368 12707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } attempt: 3
I20260806 19:48:16.170724 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.170114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49618)
0806 19:48:16.170380 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.170502 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 19:48:16.172077 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.171619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49624)
0806 19:48:16.171807 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.171911 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 19:48:16.172381 12707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:37927: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
I20260806 19:48:16.198892 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } attempt: 4
I20260806 19:48:16.200500 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.199730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49626)
0806 19:48:16.200042 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.200171 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 19:48:16.201968 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.201436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49628)
0806 19:48:16.201634 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.201735 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 19:48:16.202241 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:37927: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
W20260806 19:48:16.236732 12789 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 19:48:16.237154 12789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:48:16.237236 12789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:48:16.237329 12789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:48:16.244774 12789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:48:16.244940 12789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:48:16.245002 12789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:48:16.245055 12789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:48:16.245116 12789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:48:16.255522 12789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:44979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:39333,127.21.202.125:39149,127.21.202.124:37927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.124.keytab
--principal=kudu/127.21.202.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.124:37927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.202.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:48:16.256893 12789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:48:16.258167 12789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:48:16.270148 12795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:16.270176 12794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:16.270119 12797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:16.270942 12789 server_base.cc:1034] running on GCE node
I20260806 19:48:16.271720 12789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:48:16.273281 12789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:48:16.274569 12789 hybrid_clock.cc:648] HybridClock initialized: now 1786045696274484 us; error 47 us; skew 500 ppm
I20260806 19:48:16.274983 12789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:48:16 dist-test-slave-77v9 krb5kdc[12627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045696, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:48:16.278899 12789 init.cc:377] Logged in from keytab as kudu/127.21.202.124@KRBTEST.COM (short username kudu)
I20260806 19:48:16.280700 12789 webserver.cc:460] Webserver started at http://127.21.202.124:35063/ using document root <none> and password file <none>
I20260806 19:48:16.281592 12789 fs_manager.cc:359] Metadata directory not provided
I20260806 19:48:16.281697 12789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:48:16.281992 12789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:48:16.285444 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/data/instance:
uuid: "997dfc6357a54dc3aec707a488ef5792"
format_stamp: "Formatted at 2026-08-06 19:48:16 on dist-test-slave-77v9"
server_key: "230bf65aaae812f7c6f1f70f32a30ca0"
server_key_iv: "71263ec128f89d09c7842740827ba609"
server_key_version: "encryptionkey@0"
I20260806 19:48:16.286288 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/wal/instance:
uuid: "997dfc6357a54dc3aec707a488ef5792"
format_stamp: "Formatted at 2026-08-06 19:48:16 on dist-test-slave-77v9"
server_key: "230bf65aaae812f7c6f1f70f32a30ca0"
server_key_iv: "71263ec128f89d09c7842740827ba609"
server_key_version: "encryptionkey@0"
I20260806 19:48:16.291911 12789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:48:16.295825 12804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:48:16.297590 12789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:48:16.297765 12789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/wal
uuid: "997dfc6357a54dc3aec707a488ef5792"
format_stamp: "Formatted at 2026-08-06 19:48:16 on dist-test-slave-77v9"
server_key: "230bf65aaae812f7c6f1f70f32a30ca0"
server_key_iv: "71263ec128f89d09c7842740827ba609"
server_key_version: "encryptionkey@0"
I20260806 19:48:16.297973 12789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:48:16.326676 12789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:48:16.331302 12789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:48:16.331565 12789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:48:16.349720 12707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } attempt: 4
I20260806 19:48:16.351347 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.350879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49632)
0806 19:48:16.351093 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.351193 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 19:48:16.352775 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.352228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49636)
0806 19:48:16.352452 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.352605 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260806 19:48:16.353101 12707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:37927: Network error: Client connection negotiation failed: client connection to 127.21.202.124:37927: connect: Connection refused (error 111)
I20260806 19:48:16.353555 12789 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.124:37927
I20260806 19:48:16.353576 12855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.124:37927 every 8 connection(s)
I20260806 19:48:16.354995 12789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/data/info.pb
I20260806 19:48:16.358922 12856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:48:16.360893 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 12789
I20260806 19:48:16.361091 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786045310306758-22313-0/minicluster-data/master-2/wal/instance
I20260806 19:48:16.361652 22313 external_mini_cluster.cc:1428] Setting key 0921dc7080c238ddecdbdd251889268a
I20260806 19:48:16.367435 12856 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:48:16 dist-test-slave-77v9 krb5kdc[12627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045696, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.124@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
Aug 06 19:48:16 dist-test-slave-77v9 krb5kdc[12627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.124@KRBTEST.COM
I20260806 19:48:16.388733 12860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.370002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55626 (local address 127.21.202.126:39333)
0806 19:48:16.370581 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:16.370589 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:16.370869 (+ 280us) server_negotiation.cc:407] Connection header received
0806 19:48:16.371118 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:16.371125 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:16.371231 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:16.371375 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:16.372596 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.373891 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.375461 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.376119 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.376242 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.380893 (+ 4651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:16.380901 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:16.380905 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:16.380960 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:16.384460 (+ 3500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.385511 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.385538 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.385542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.385758 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.386467 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.386474 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.386478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.387001 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:16.387189 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:16.387880 (+ 691us) server_negotiation.cc:299] Negotiation successful
0806 19:48:16.388145 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":301,"threads_started":1}
I20260806 19:48:16.389582 12861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.369972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:39333 (local address 127.0.0.1:55626)
0806 19:48:16.370701 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.370748 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:16.370923 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:16.371412 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:16.371417 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:16.372082 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:16.372405 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:16.372418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.374090 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.374094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:16.375267 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:16.375282 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.376372 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.376381 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:16.376545 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.377176 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:16.377198 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:16.380357 (+ 3159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:16.384633 (+ 4276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:16.384647 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:16.384664 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:16.385335 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:16.385922 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:16.385967 (+ 45us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:16.385973 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:16.386271 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:16.387175 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:16.387187 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:16.387446 (+ 259us) client_negotiation.cc:769] Sending connection context
0806 19:48:16.387767 (+ 321us) client_negotiation.cc:240] Negotiation successful
0806 19:48:16.388652 (+ 885us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":313,"thread_start_us":350,"threads_started":1}
I20260806 19:48:16.389600 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.363893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.21.202.124:37927)
0806 19:48:16.364949 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:16.364963 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:16.365023 (+ 60us) server_negotiation.cc:407] Connection header received
0806 19:48:16.367039 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:16.367066 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:16.367534 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:16.368029 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:16.370551 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.372428 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.374532 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.375405 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.375644 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.380894 (+ 5250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:16.380913 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:16.380928 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:16.380965 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:16.384612 (+ 3647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.385687 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.385703 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.385708 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.385986 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.386570 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.386577 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.386582 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.387216 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:16.387567 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:16.387898 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 19:48:16.388274 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":706,"thread_start_us":411,"threads_started":1}
I20260806 19:48:16.391484 12856 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:48:16 dist-test-slave-77v9 krb5kdc[12627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045696, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.124@KRBTEST.COM for kudu/127.21.202.125@KRBTEST.COM
Aug 06 19:48:16 dist-test-slave-77v9 krb5kdc[12627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:48:16.409068 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.392736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52176 (local address 127.21.202.125:39149)
0806 19:48:16.392965 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:16.392969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:16.392988 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:48:16.393061 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:16.393066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:16.393145 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:16.393258 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:16.394280 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.395393 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.397084 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.397744 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.397914 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.401822 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:16.401830 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:16.401835 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:16.401892 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:16.405777 (+ 3885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.406617 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.406623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.406627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.406892 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.407342 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.407349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.407354 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.407760 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:16.407941 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:16.408387 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 19:48:16.408618 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260806 19:48:16.409054 12861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.392436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:39149 (local address 127.0.0.1:52176)
0806 19:48:16.392678 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.392724 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:16.392905 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:16.393313 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:16.393322 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:16.393909 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:16.394107 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:16.394118 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.395543 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.395549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:16.396948 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:16.396961 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.397941 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.397969 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:16.398152 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.398800 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:16.398826 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:16.401536 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:16.405981 (+ 4445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:16.405986 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:16.405989 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:16.406443 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:16.407039 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:16.407043 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:16.407046 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:16.407205 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:16.407899 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:16.407905 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:16.408095 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 19:48:16.408254 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 19:48:16.408594 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 19:48:16.410193 12856 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:48:16.410776 12860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.393287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55638 (local address 127.21.202.126:39333)
0806 19:48:16.393487 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:16.393493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:16.393515 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:48:16.393596 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:16.393602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:16.393701 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:16.393828 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:16.395406 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.396533 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.398264 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.398931 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.399082 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.403671 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:16.403681 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:16.403687 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:16.403744 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:16.407519 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.408328 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.408342 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.408344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.408488 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.408941 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.408947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.408950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.409332 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:16.409637 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:16.410200 (+ 563us) server_negotiation.cc:299] Negotiation successful
0806 19:48:16.410441 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 19:48:16 dist-test-slave-77v9 krb5kdc[12627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045696, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.124@KRBTEST.COM for kudu/127.21.202.124@KRBTEST.COM
Aug 06 19:48:16 dist-test-slave-77v9 krb5kdc[12627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.125@KRBTEST.COM
I20260806 19:48:16.425635 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.411048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49650 (local address 127.21.202.124:37927)
0806 19:48:16.411191 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:16.411195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:16.411358 (+ 163us) server_negotiation.cc:407] Connection header received
0806 19:48:16.411493 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:16.411500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:16.411579 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:16.411690 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:16.412604 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.413691 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.415142 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.415540 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.415699 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.419086 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:16.419093 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:16.419098 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:16.419139 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:16.422571 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.423384 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.423389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.423394 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.423539 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.424020 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.424025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.424029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.424356 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:16.424522 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:16.424928 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 19:48:16.425195 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":40}
I20260806 19:48:16.425729 12861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.411005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49650)
0806 19:48:16.411262 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.411281 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:16.411391 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:16.411771 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:16.411776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:16.412318 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:16.412472 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:16.412481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.413855 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.413859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:16.414990 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:16.415006 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.415710 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.415715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:16.415874 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.416505 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:16.416522 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:16.418879 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:16.422720 (+ 3841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:16.422726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:16.422731 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:16.423254 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:16.423684 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:16.423688 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:16.423691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:16.423871 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:16.424480 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:16.424485 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:16.424635 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 19:48:16.424817 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 19:48:16.425349 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":168}
I20260806 19:48:16.429095 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.412666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52182 (local address 127.21.202.125:39149)
0806 19:48:16.412879 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:16.412885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:16.412929 (+ 44us) server_negotiation.cc:407] Connection header received
0806 19:48:16.413092 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:16.413096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:16.413152 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:16.413231 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:16.414600 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.415505 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.416999 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.417663 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.417815 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.423152 (+ 5337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:16.423158 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:16.423161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:16.423197 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:16.426271 (+ 3074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.427092 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.427098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.427100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.427247 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.427645 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.427650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.427652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.427913 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:16.428210 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:16.428511 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 19:48:16.428744 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 19:48:16.431665 12856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792: Bootstrap starting.
I20260806 19:48:16.434696 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:48:16.435741 12856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792: Neither blocks nor log segments found. Creating new log.
I20260806 19:48:16.436962 12856 log.cc:824] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792: Log is configured to *not* fsync() on all Append() calls
I20260806 19:48:16.440176 12856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792: No bootstrap required, opened a new log
I20260806 19:48:16.437693 12682 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "eba2682bfd684ad582ea647149b351eb"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55638
I20260806 19:48:16.448295 12856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } }
I20260806 19:48:16.448707 12856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997dfc6357a54dc3aec707a488ef5792, State: Initialized, Role: FOLLOWER
I20260806 19:48:16.449255 12856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } }
I20260806 19:48:16.450392 12863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } } }
I20260806 19:48:16.450791 12863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:48:16.451534 12856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 19:48:16.462097 12874 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:48:16.462327 12874 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:48:16.459760 12789 master_runner.cc:186] Error getting master registration for 127.21.202.126:39333: OK, instance_id { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" instance_seqno: 1786045695200880 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 19:48:16.471848 12789 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 19:48:16.501489 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } attempt: 5
Aug 06 19:48:16 dist-test-slave-77v9 krb5kdc[12627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045695, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.125@KRBTEST.COM for kudu/127.21.202.124@KRBTEST.COM
I20260806 19:48:16.517968 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.502628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49654 (local address 127.21.202.124:37927)
0806 19:48:16.502850 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:16.502857 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:16.502876 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:48:16.502945 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:16.502950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:16.503036 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:16.503135 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:16.503945 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.504760 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.505998 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.506522 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.506664 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.510481 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:16.510490 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:16.510495 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:16.510543 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:16.514386 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.515283 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.515289 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.515292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.515435 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.515921 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.515926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.515929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.516566 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:16.516759 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:16.517233 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 19:48:16.517486 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:48:16.517956 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.502365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49654)
0806 19:48:16.502558 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.502585 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:16.502717 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:16.503191 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:16.503196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:16.503620 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:16.503783 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:16.503792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.504946 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.504954 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:16.505837 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:16.505850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.506684 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.506690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:16.506947 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.507620 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:16.507637 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:16.510199 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:16.514567 (+ 4368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:16.514575 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:16.514581 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:16.515148 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:16.515558 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:16.515571 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:16.515575 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:16.515796 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:16.516712 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:16.516718 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:16.516928 (+ 210us) client_negotiation.cc:769] Sending connection context
0806 19:48:16.517112 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 19:48:16.517486 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 19:48:16.524933 12781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d: Bootstrap starting.
I20260806 19:48:16.529268 12781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d: Neither blocks nor log segments found. Creating new log.
I20260806 19:48:16.530782 12781 log.cc:824] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d: Log is configured to *not* fsync() on all Append() calls
I20260806 19:48:16.534792 12781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d: No bootstrap required, opened a new log
I20260806 19:48:16.548360 12781 raft_consensus.cc:348] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } }
I20260806 19:48:16.548972 12781 raft_consensus.cc:729] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a18b062fa7bc429db9910366a751049d, State: Initialized, Role: FOLLOWER
I20260806 19:48:16.549628 12781 consensus_queue.cc:260] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } }
I20260806 19:48:16.550789 12877 sys_catalog.cc:455] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } } }
I20260806 19:48:16.551608 12877 sys_catalog.cc:458] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:48:16.552340 12781 sys_catalog.cc:564] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d [sys.catalog]: configured and running, proceeding with master startup.
W20260806 19:48:16.565426 12888 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:48:16.565616 12888 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:48:16.562337 12714 master_runner.cc:186] Error getting master registration for 127.21.202.126:39333: OK, instance_id { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" instance_seqno: 1786045695200880 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 19:48:16.575878 12714 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 19:48:16.583897 12810 tablet.cc:2369] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:48:16.630321 12707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } attempt: 5
Aug 06 19:48:16 dist-test-slave-77v9 krb5kdc[12627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045695, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for kudu/127.21.202.124@KRBTEST.COM
I20260806 19:48:16.650486 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.631181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:37927 (local address 127.0.0.1:49660)
0806 19:48:16.631410 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:16.631432 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:16.631572 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:16.632452 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:16.632459 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:16.633145 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:16.633380 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:16.633395 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.634881 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.634888 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:16.636542 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:16.636562 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.637931 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.637938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:16.638150 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.638930 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:16.638957 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:16.642515 (+ 3558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:16.647422 (+ 4907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:16.647429 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:16.647434 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:16.648025 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:16.648518 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:16.648522 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:16.648526 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:16.648708 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:16.649395 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:16.649407 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:16.649589 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 19:48:16.649754 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 19:48:16.650068 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 19:48:16.650581 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.631680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49660 (local address 127.21.202.124:37927)
0806 19:48:16.631992 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:16.631999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:16.632023 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:48:16.632089 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:16.632094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:16.632204 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:16.632317 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:16.633592 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.634743 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.636731 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.637520 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.637950 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.642791 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:16.642800 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:16.642805 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:16.642854 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:16.647206 (+ 4352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.648176 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.648183 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.648188 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.648391 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.648867 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.648873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.648877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.649266 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:16.649445 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:16.649884 (+ 439us) server_negotiation.cc:299] Negotiation successful
0806 19:48:16.650161 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 19:48:16.658504 12707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb: Bootstrap starting.
I20260806 19:48:16.663239 12707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb: Neither blocks nor log segments found. Creating new log.
I20260806 19:48:16.665273 12707 log.cc:824] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb: Log is configured to *not* fsync() on all Append() calls
I20260806 19:48:16.670856 12707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb: No bootstrap required, opened a new log
I20260806 19:48:16.684082 12707 raft_consensus.cc:348] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } }
I20260806 19:48:16.684996 12707 raft_consensus.cc:729] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba2682bfd684ad582ea647149b351eb, State: Initialized, Role: FOLLOWER
I20260806 19:48:16.685907 12707 consensus_queue.cc:260] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } }
I20260806 19:48:16.687496 12891 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } } }
I20260806 19:48:16.687963 12891 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [sys.catalog]: This master's current role is: FOLLOWER
W20260806 19:48:16.689363 12661 tablet.cc:2369] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:48:16.691104 12682 raft_consensus.cc:482] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 19:48:16.691320 12682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:48:16.691504 12707 sys_catalog.cc:564] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:48:16.694945 12682 raft_consensus.cc:504] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } }
I20260806 19:48:16.697647 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eba2682bfd684ad582ea647149b351eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a18b062fa7bc429db9910366a751049d"
I20260806 19:48:16.698170 12756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:48:16.701956 12756 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eba2682bfd684ad582ea647149b351eb in term 1.
I20260806 19:48:16.702796 12658 leader_election.cc:302] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a18b062fa7bc429db9910366a751049d, eba2682bfd684ad582ea647149b351eb; no voters:
I20260806 19:48:16.703303 12891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:48:16.704625 12831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eba2682bfd684ad582ea647149b351eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "997dfc6357a54dc3aec707a488ef5792"
I20260806 19:48:16.705199 12831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:48:16.709187 12831 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eba2682bfd684ad582ea647149b351eb in term 1.
I20260806 19:48:16.709991 12682 leader_election.cc:288] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [CANDIDATE]: Term 1 election: Requested vote from peers a18b062fa7bc429db9910366a751049d (127.21.202.125:39149), 997dfc6357a54dc3aec707a488ef5792 (127.21.202.124:37927)
I20260806 19:48:16.722828 12891 raft_consensus.cc:686] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [term 1 LEADER]: Becoming Leader. State: Replica: eba2682bfd684ad582ea647149b351eb, State: Running, Role: LEADER
I20260806 19:48:16.723632 12891 consensus_queue.cc:237] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } }
I20260806 19:48:16.738260 12893 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba2682bfd684ad582ea647149b351eb. Latest consensus state: current_term: 1 leader_uuid: "eba2682bfd684ad582ea647149b351eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } } }
I20260806 19:48:16.738904 12893 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:16.745563 12902 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260806 19:48:16.752424 12735 tablet.cc:2369] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:48:16.772289 12902 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:48:16.808043 12831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792 [term 1 FOLLOWER]: Refusing update from remote peer eba2682bfd684ad582ea647149b351eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:48:16.810972 12756 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d [term 1 FOLLOWER]: Refusing update from remote peer eba2682bfd684ad582ea647149b351eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:48:16.811058 12893 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:48:16.813328 12893 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:48:16.815212 12860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.732393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55652 (local address 127.21.202.126:39333)
0806 19:48:16.732581 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:16.732588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:16.745747 (+ 13159us) server_negotiation.cc:407] Connection header received
0806 19:48:16.748556 (+ 2809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:16.748563 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:16.748654 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:16.748734 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:16.758235 (+ 9501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.761000 (+ 2765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.771007 (+ 10007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.771695 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.771802 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.785494 (+ 13692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:16.785503 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:16.785511 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:16.785587 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:16.790449 (+ 4862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.802510 (+ 12061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.802520 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.802524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.802711 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.805058 (+ 2347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.805070 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.805076 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.805566 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:16.805878 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:16.814533 (+ 8655us) server_negotiation.cc:299] Negotiation successful
0806 19:48:16.814822 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 19:48:16.816854 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.735122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52186 (local address 127.21.202.125:39149)
0806 19:48:16.735298 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:16.735305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:16.752553 (+ 17248us) server_negotiation.cc:407] Connection header received
0806 19:48:16.753890 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:16.753897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:16.753986 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:16.754059 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:16.764648 (+ 10589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.766002 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.773332 (+ 7330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.774111 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.774231 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.791704 (+ 17473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:16.791716 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:16.791723 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:16.791778 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:16.796149 (+ 4371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.803578 (+ 7429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.803587 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.803590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.803782 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.806491 (+ 2709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.806497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.806502 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.812162 (+ 5660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:16.812468 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:16.816149 (+ 3681us) server_negotiation.cc:299] Negotiation successful
0806 19:48:16.816432 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 19:48:16.818356 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:16.746353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49664 (local address 127.21.202.124:37927)
0806 19:48:16.746727 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:16.746739 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:16.753555 (+ 6816us) server_negotiation.cc:407] Connection header received
0806 19:48:16.755838 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:16.755844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:16.755931 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:16.756013 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:16.768008 (+ 11995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.769080 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.778162 (+ 9082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:16.778942 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:16.779079 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:16.797563 (+ 18484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:16.797571 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:16.797576 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:16.797625 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:16.801298 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.804315 (+ 3017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.804322 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.804327 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.804518 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:16.813148 (+ 8630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:16.813155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:16.813159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:16.813618 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:16.813900 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:16.817646 (+ 3746us) server_negotiation.cc:299] Negotiation successful
0806 19:48:16.817947 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231}
I20260806 19:48:16.825519 12877 sys_catalog.cc:455] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba2682bfd684ad582ea647149b351eb. Latest consensus state: current_term: 1 leader_uuid: "eba2682bfd684ad582ea647149b351eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } } }
I20260806 19:48:16.825888 12877 sys_catalog.cc:458] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:48:16.829027 12911 mvcc.cc:205] Tried to move back new op lower bound from 7315643174077022208 to 7315643173786972160. Current Snapshot: MvccSnapshot[applied={T|T < 7315643174077022208}]
I20260806 19:48:16.830089 12915 mvcc.cc:205] Tried to move back new op lower bound from 7315643174077022208 to 7315643173786972160. Current Snapshot: MvccSnapshot[applied={T|T < 7315643174077022208}]
I20260806 19:48:16.840565 12877 sys_catalog.cc:455] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eba2682bfd684ad582ea647149b351eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } } }
I20260806 19:48:16.840909 12877 sys_catalog.cc:458] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:48:16.843523 12893 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eba2682bfd684ad582ea647149b351eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } } }
I20260806 19:48:16.843791 12893 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:16.844637 12902 catalog_manager.cc:1347] Generated new cluster ID: 58296d39cf5f46e186d52b51cfd24a64
I20260806 19:48:16.844738 12902 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:48:16.853440 12891 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eba2682bfd684ad582ea647149b351eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } } }
I20260806 19:48:16.853824 12891 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:16.854447 12863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba2682bfd684ad582ea647149b351eb. Latest consensus state: current_term: 1 leader_uuid: "eba2682bfd684ad582ea647149b351eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } } }
I20260806 19:48:16.854771 12863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:48:16.857774 12916 mvcc.cc:205] Tried to move back new op lower bound from 7315643174077022208 to 7315643173786972160. Current Snapshot: MvccSnapshot[applied={T|T < 7315643174077022208}]
I20260806 19:48:16.862478 12863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eba2682bfd684ad582ea647149b351eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2682bfd684ad582ea647149b351eb" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 39333 } } peers { permanent_uuid: "a18b062fa7bc429db9910366a751049d" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 39149 } } peers { permanent_uuid: "997dfc6357a54dc3aec707a488ef5792" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 37927 } } }
I20260806 19:48:16.862775 12863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:48:16.889191 12902 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:48:16.891932 12902 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:48:16.918025 12902 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eba2682bfd684ad582ea647149b351eb: Generated new TSK 0
I20260806 19:48:16.919688 12902 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 19:48:17.047077 12860 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:17.033681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55658 (local address 127.21.202.126:39333)
0806 19:48:17.033906 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.033912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.044940 (+ 11028us) server_negotiation.cc:407] Connection header received
0806 19:48:17.045279 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.045288 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.045390 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.045470 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:17.046795 (+ 1325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55658: BlockingRecv error: recv got EOF from 127.0.0.1:55658 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 19:48:17.049717 12859 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:17.046288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49678 (local address 127.21.202.124:37927)
0806 19:48:17.046511 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.046519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.047906 (+ 1387us) server_negotiation.cc:407] Connection header received
0806 19:48:17.048329 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.048335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.048414 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.048485 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:17.049470 (+ 985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49678: BlockingRecv error: recv got EOF from 127.0.0.1:49678 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 19:48:17.054306 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:17.033274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52196 (local address 127.21.202.125:39149)
0806 19:48:17.033630 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.033638 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.045539 (+ 11901us) server_negotiation.cc:407] Connection header received
0806 19:48:17.045718 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.045723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.045827 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.045991 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:17.053988 (+ 7997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52196: BlockingRecv error: recv got EOF from 127.0.0.1:52196 (error 108)
Metrics: {"server-negotiator.queue_time_us":199}
W20260806 19:48:17.075433 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:17.068063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52206 (local address 127.21.202.125:39149)
0806 19:48:17.068267 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.068274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.073577 (+ 5303us) server_negotiation.cc:407] Connection header received
0806 19:48:17.073890 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.073897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.074054 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.074167 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:17.075130 (+ 963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52206: BlockingRecv error: recv got EOF from 127.0.0.1:52206 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 19:48:17.082443 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:17.080169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52214 (local address 127.21.202.125:39149)
0806 19:48:17.080353 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.080364 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.080882 (+ 518us) server_negotiation.cc:407] Connection header received
0806 19:48:17.081103 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.081109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.081206 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.081336 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:17.082208 (+ 872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52214: BlockingRecv error: recv got EOF from 127.0.0.1:52214 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 19:48:17.090550 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:17.088174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52220 (local address 127.21.202.125:39149)
0806 19:48:17.088360 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.088366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.088907 (+ 541us) server_negotiation.cc:407] Connection header received
0806 19:48:17.089145 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.089152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.089257 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.089388 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:17.090259 (+ 871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52220: BlockingRecv error: recv got EOF from 127.0.0.1:52220 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 19:48:17.104303 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:17.101733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52232 (local address 127.21.202.125:39149)
0806 19:48:17.101960 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.101967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.102548 (+ 581us) server_negotiation.cc:407] Connection header received
0806 19:48:17.102774 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.102786 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.102929 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.103055 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:17.104007 (+ 952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52232: BlockingRecv error: recv got EOF from 127.0.0.1:52232 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260806 19:48:17.121253 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:17.118545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52238 (local address 127.21.202.125:39149)
0806 19:48:17.118787 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.118794 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.119412 (+ 618us) server_negotiation.cc:407] Connection header received
0806 19:48:17.119746 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.119752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.119852 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.119977 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:17.120966 (+ 989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52238: BlockingRecv error: recv got EOF from 127.0.0.1:52238 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 19:48:17.143899 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:17.141478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52244 (local address 127.21.202.125:39149)
0806 19:48:17.141692 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.141698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.142227 (+ 529us) server_negotiation.cc:407] Connection header received
0806 19:48:17.142444 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.142454 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.142558 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.142684 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:17.143604 (+ 920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52244: BlockingRecv error: recv got EOF from 127.0.0.1:52244 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260806 19:48:17.185125 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:17.182770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52250 (local address 127.21.202.125:39149)
0806 19:48:17.182949 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.182956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.183509 (+ 553us) server_negotiation.cc:407] Connection header received
0806 19:48:17.183714 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.183722 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.183842 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.183960 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:17.184840 (+ 880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52250: BlockingRecv error: recv got EOF from 127.0.0.1:52250 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 19:48:17.255735 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:17.253013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52262 (local address 127.21.202.125:39149)
0806 19:48:17.253275 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.253298 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.253962 (+ 664us) server_negotiation.cc:407] Connection header received
0806 19:48:17.254236 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.254242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.254346 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.254482 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:17.255443 (+ 961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52262: BlockingRecv error: recv got EOF from 127.0.0.1:52262 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260806 19:48:17.389891 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:17.387649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52274 (local address 127.21.202.125:39149)
0806 19:48:17.387962 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.387972 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.388323 (+ 351us) server_negotiation.cc:407] Connection header received
0806 19:48:17.388589 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.388594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.388675 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.388795 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:17.389682 (+ 887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52274: BlockingRecv error: recv got EOF from 127.0.0.1:52274 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 19:48:17.464391 12874 catalog_manager.cc:1259] Loaded cluster ID: 58296d39cf5f46e186d52b51cfd24a64
I20260806 19:48:17.464538 12874 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792: loading cluster ID for follower catalog manager: success
I20260806 19:48:17.467175 12874 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792: acquiring CA information for follower catalog manager: success
I20260806 19:48:17.469134 12874 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 997dfc6357a54dc3aec707a488ef5792: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:48:17.567703 12888 catalog_manager.cc:1259] Loaded cluster ID: 58296d39cf5f46e186d52b51cfd24a64
I20260806 19:48:17.567937 12888 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d: loading cluster ID for follower catalog manager: success
I20260806 19:48:17.570487 12888 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d: acquiring CA information for follower catalog manager: success
I20260806 19:48:17.572211 12888 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a18b062fa7bc429db9910366a751049d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:48:17.658517 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:17.650456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52288 (local address 127.21.202.125:39149)
0806 19:48:17.650718 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.650723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.651100 (+ 377us) server_negotiation.cc:407] Connection header received
0806 19:48:17.651419 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.651423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.651690 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.651856 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:48:17.652505 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:17.653573 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:17.655369 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:17.655836 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:17.656181 (+ 345us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:17.656754 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:48:17.657705 (+ 951us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:48:17.657866 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:17.658062 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 19:48:17.658155 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 19:48:17.671357 12987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:17.665214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49694 (local address 127.21.202.124:37927)
0806 19:48:17.665690 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.665694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.665789 (+ 95us) server_negotiation.cc:407] Connection header received
0806 19:48:17.665979 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.665982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.666145 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.666286 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:48:17.666719 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:17.667548 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:17.669020 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:17.669432 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:17.669539 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:17.670155 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:48:17.670737 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:48:17.670860 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:17.671006 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 19:48:17.671083 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":286,"threads_started":1}
I20260806 19:48:17.686817 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:17.678679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52298 (local address 127.21.202.125:39149)
0806 19:48:17.678906 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.678941 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.679889 (+ 948us) server_negotiation.cc:407] Connection header received
0806 19:48:17.680146 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.680153 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.680249 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.680313 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:48:17.681220 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:17.682193 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:17.684677 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:17.685380 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:17.685499 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:17.685984 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:48:17.686131 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:48:17.686228 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:17.686386 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 19:48:17.686485 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 19:48:17.687224 12994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:17.678679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55664 (local address 127.21.202.126:39333)
0806 19:48:17.679310 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.679317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.679783 (+ 466us) server_negotiation.cc:407] Connection header received
0806 19:48:17.680149 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.680154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.680394 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.680558 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:48:17.681476 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:17.682398 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:17.684784 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:17.685253 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:17.685390 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:17.685920 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:48:17.686335 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:48:17.686429 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:17.686853 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 19:48:17.686943 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":393,"threads_started":1}
I20260806 19:48:17.688012 12987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:17.680005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49698 (local address 127.21.202.124:37927)
0806 19:48:17.680237 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:17.680246 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:17.680862 (+ 616us) server_negotiation.cc:407] Connection header received
0806 19:48:17.681129 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:17.681155 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:17.681240 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:17.681339 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:48:17.681902 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:17.682750 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:17.684606 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:17.685278 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:17.685402 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:17.686386 (+ 984us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:48:17.686517 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:48:17.686606 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:17.687729 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0806 19:48:17.687813 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 19:48:17.693315 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 12640
I20260806 19:48:17.694729 12702 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:48:17.811755 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 12640
W20260806 19:48:17.835402 12806 connection.cc:535] server connection from 127.0.0.1:49660 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 19:48:17.835814 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 12714
I20260806 19:48:17.837018 12776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:48:17.948194 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 12714
I20260806 19:48:17.971216 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 12789
I20260806 19:48:17.972462 12851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:48:18.083351 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 12789
2026-08-06T19:48:18Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4907 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:48:18 dist-test-slave-77v9 krb5kdc[13001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:48:18 dist-test-slave-77v9 krb5kdc[13001](info): set up 2 sockets
Aug 06 19:48:18 dist-test-slave-77v9 krb5kdc[13001](info): commencing operation
krb5kdc: starting...
W20260806 19:48:19.297750 22313 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/test-user.keytab.
Aug 06 19:48:19 dist-test-slave-77v9 krb5kdc[13001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:48:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:48:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
Entry for principal kudu/127.21.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab.
I20260806 19:48:19.653465 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46071
--webserver_interface=127.21.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:41465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:46071,127.21.202.125:33459,127.21.202.124:43819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:48:20.022183 13014 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 19:48:20.022596 13014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:48:20.022727 13014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:48:20.022862 13014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:48:20.030525 13014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:48:20.030650 13014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:48:20.030774 13014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:48:20.030858 13014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:48:20.030967 13014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:48:20.041412 13014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:41465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:46071,127.21.202.125:33459,127.21.202.124:43819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.126.keytab
--principal=kudu/127.21.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.126:46071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:48:20.042891 13014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:48:20.044265 13014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:48:20.056759 13019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:20.056825 13022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:20.056695 13020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:20.057919 13014 server_base.cc:1034] running on GCE node
I20260806 19:48:20.058869 13014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:48:20.060477 13014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:48:20.061797 13014 hybrid_clock.cc:648] HybridClock initialized: now 1786045700061704 us; error 41 us; skew 500 ppm
I20260806 19:48:20.062184 13014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:48:20 dist-test-slave-77v9 krb5kdc[13001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045700, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:48:20.065927 13014 init.cc:377] Logged in from keytab as kudu/127.21.202.126@KRBTEST.COM (short username kudu)
I20260806 19:48:20.067966 13014 webserver.cc:460] Webserver started at http://127.21.202.126:44937/ using document root <none> and password file <none>
I20260806 19:48:20.068742 13014 fs_manager.cc:359] Metadata directory not provided
I20260806 19:48:20.068897 13014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:48:20.069254 13014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:48:20.072685 13014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/data/instance:
uuid: "a1ac208b80a64adaae648ace12a63f35"
format_stamp: "Formatted at 2026-08-06 19:48:20 on dist-test-slave-77v9"
server_key: "370ca0594ac5de222d6cc358f5a0cd75"
server_key_iv: "91ec08223af06f2af35fd6438b5a531a"
server_key_version: "encryptionkey@0"
I20260806 19:48:20.073653 13014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/wal/instance:
uuid: "a1ac208b80a64adaae648ace12a63f35"
format_stamp: "Formatted at 2026-08-06 19:48:20 on dist-test-slave-77v9"
server_key: "370ca0594ac5de222d6cc358f5a0cd75"
server_key_iv: "91ec08223af06f2af35fd6438b5a531a"
server_key_version: "encryptionkey@0"
I20260806 19:48:20.079293 13014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:48:20.083261 13029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:48:20.084942 13014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:48:20.085170 13014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/wal
uuid: "a1ac208b80a64adaae648ace12a63f35"
format_stamp: "Formatted at 2026-08-06 19:48:20 on dist-test-slave-77v9"
server_key: "370ca0594ac5de222d6cc358f5a0cd75"
server_key_iv: "91ec08223af06f2af35fd6438b5a531a"
server_key_version: "encryptionkey@0"
I20260806 19:48:20.085448 13014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:48:20.120278 13014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:48:20.126474 13014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:48:20.126832 13014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:48:20.149546 13014 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.126:46071
I20260806 19:48:20.149561 13080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.126:46071 every 8 connection(s)
I20260806 19:48:20.151149 13014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/data/info.pb
I20260806 19:48:20.155331 13081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:48:20.159101 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 13014
I20260806 19:48:20.159389 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-0/wal/instance
I20260806 19:48:20.159914 22313 external_mini_cluster.cc:1428] Setting key 1d268a7360eff4080746e972df8ae75f
I20260806 19:48:20.163096 13081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:48:20 dist-test-slave-77v9 krb5kdc[13001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045700, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:48:20.191852 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.166547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46071 (local address 127.0.0.1:39410)
0806 19:48:20.167423 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.167474 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:20.169830 (+ 2356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:20.170939 (+ 1109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:20.170944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:20.171550 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:20.171897 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:20.171914 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.174389 (+ 2475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.174473 (+ 84us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:20.176453 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:20.176473 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.177698 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.177709 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:20.177992 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:20.178785 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:20.178820 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:20.181875 (+ 3055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:20.187054 (+ 5179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:20.187066 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:20.187087 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:20.187548 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:20.188149 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:20.188153 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:20.188155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:20.188311 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:20.189452 (+ 1141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:20.189459 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:20.189721 (+ 262us) client_negotiation.cc:769] Sending connection context
0806 19:48:20.189980 (+ 259us) client_negotiation.cc:240] Negotiation successful
0806 19:48:20.190771 (+ 791us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":207,"thread_start_us":384,"threads_started":1}
I20260806 19:48:20.191895 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.167892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39410 (local address 127.21.202.126:46071)
0806 19:48:20.168716 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:20.168728 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:20.168802 (+ 74us) server_negotiation.cc:407] Connection header received
0806 19:48:20.170274 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:20.170284 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:20.170642 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:20.170867 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:20.172126 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.174186 (+ 2060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.176650 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.177415 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.177590 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:20.182189 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:20.182204 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:20.182227 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:20.182283 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:20.186856 (+ 4573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:20.187706 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:20.187719 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:20.187725 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:20.187972 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:20.188423 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:20.188430 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:20.188436 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:20.189278 (+ 842us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:20.189486 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:20.190160 (+ 674us) server_negotiation.cc:299] Negotiation successful
0806 19:48:20.190532 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":354,"threads_started":1}
I20260806 19:48:20.195468 13081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:48:20.197129 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.196423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33459 (local address 127.0.0.1:57986)
0806 19:48:20.196772 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.196989 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228}
WARNING: no policy specified for kudu/127.21.202.125@KRBTEST.COM; defaulting to no policy
W20260806 19:48:20.197455 13030 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
I20260806 19:48:20.199255 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.198731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33459 (local address 127.0.0.1:57988)
0806 19:48:20.198903 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.199080 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 19:48:20.199491 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:33459: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.125.keytab.
Entry for principal kudu/127.21.202.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.125.keytab.
I20260806 19:48:20.241276 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.125:33459
--webserver_interface=127.21.202.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.125.keytab
--principal=kudu/127.21.202.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:41465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:46071,127.21.202.125:33459,127.21.202.124:43819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:48:20.248901 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } attempt: 1
I20260806 19:48:20.250707 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.249930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33459 (local address 127.0.0.1:57998)
0806 19:48:20.250288 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.250502 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260806 19:48:20.251835 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.251404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33459 (local address 127.0.0.1:58014)
0806 19:48:20.251582 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.251682 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260806 19:48:20.252138 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:33459: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
I20260806 19:48:20.320508 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } attempt: 2
I20260806 19:48:20.321995 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.321351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33459 (local address 127.0.0.1:58018)
0806 19:48:20.321644 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.321781 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 19:48:20.323673 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.323037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33459 (local address 127.0.0.1:58022)
0806 19:48:20.323282 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.323484 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260806 19:48:20.324020 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:33459: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
I20260806 19:48:20.415518 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } attempt: 3
I20260806 19:48:20.416913 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.416342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33459 (local address 127.0.0.1:58030)
0806 19:48:20.416581 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.416700 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 19:48:20.418098 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.417680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33459 (local address 127.0.0.1:58042)
0806 19:48:20.417836 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.417926 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 19:48:20.418327 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:33459: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
I20260806 19:48:20.561625 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } attempt: 4
I20260806 19:48:20.563073 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.562397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33459 (local address 127.0.0.1:58048)
0806 19:48:20.562777 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.562892 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260806 19:48:20.564343 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.563881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33459 (local address 127.0.0.1:58056)
0806 19:48:20.564043 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.564150 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 19:48:20.564556 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.125:33459: Network error: Client connection negotiation failed: client connection to 127.21.202.125:33459: connect: Connection refused (error 111)
W20260806 19:48:20.617304 13088 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 19:48:20.617671 13088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:48:20.617796 13088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:48:20.617890 13088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:48:20.625557 13088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:48:20.625674 13088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:48:20.625789 13088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:48:20.625873 13088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:48:20.625979 13088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:48:20.637974 13088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:41465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:46071,127.21.202.125:33459,127.21.202.124:43819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.125.keytab
--principal=kudu/127.21.202.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.125:33459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.202.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:48:20.639425 13088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:48:20.640853 13088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:48:20.653872 13096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:20.653923 13094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:20.653934 13093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:20.655872 13088 server_base.cc:1034] running on GCE node
I20260806 19:48:20.656716 13088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:48:20.658373 13088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:48:20.659624 13088 hybrid_clock.cc:648] HybridClock initialized: now 1786045700659563 us; error 29 us; skew 500 ppm
I20260806 19:48:20.660095 13088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:48:20 dist-test-slave-77v9 krb5kdc[13001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045700, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:48:20.663623 13088 init.cc:377] Logged in from keytab as kudu/127.21.202.125@KRBTEST.COM (short username kudu)
I20260806 19:48:20.665683 13088 webserver.cc:460] Webserver started at http://127.21.202.125:44953/ using document root <none> and password file <none>
I20260806 19:48:20.666491 13088 fs_manager.cc:359] Metadata directory not provided
I20260806 19:48:20.666635 13088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:48:20.667001 13088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:48:20.670251 13088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/data/instance:
uuid: "287aefad5b6043eebf365ff4d2ad8739"
format_stamp: "Formatted at 2026-08-06 19:48:20 on dist-test-slave-77v9"
server_key: "465400475cebe3f5b491856f435b9f3d"
server_key_iv: "b16fd692127e2cab91f3455029efd925"
server_key_version: "encryptionkey@0"
I20260806 19:48:20.671094 13088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/wal/instance:
uuid: "287aefad5b6043eebf365ff4d2ad8739"
format_stamp: "Formatted at 2026-08-06 19:48:20 on dist-test-slave-77v9"
server_key: "465400475cebe3f5b491856f435b9f3d"
server_key_iv: "b16fd692127e2cab91f3455029efd925"
server_key_version: "encryptionkey@0"
I20260806 19:48:20.676671 13088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 19:48:20.680958 13103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:48:20.682466 13088 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:48:20.682695 13088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/wal
uuid: "287aefad5b6043eebf365ff4d2ad8739"
format_stamp: "Formatted at 2026-08-06 19:48:20 on dist-test-slave-77v9"
server_key: "465400475cebe3f5b491856f435b9f3d"
server_key_iv: "b16fd692127e2cab91f3455029efd925"
server_key_version: "encryptionkey@0"
I20260806 19:48:20.682963 13088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:48:20.723477 13088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:48:20.728422 13088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:48:20.728766 13088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:48:20.749241 13088 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.125:33459
I20260806 19:48:20.749285 13154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.125:33459 every 8 connection(s)
I20260806 19:48:20.750708 13088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/data/info.pb
I20260806 19:48:20.754848 13155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:48:20.759292 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 13088
I20260806 19:48:20.759521 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-1/wal/instance
I20260806 19:48:20.760205 22313 external_mini_cluster.cc:1428] Setting key 6c7e2a6d76c1c9df9ebbaf456971b517
I20260806 19:48:20.764484 13155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:48:20 dist-test-slave-77v9 krb5kdc[13001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045700, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.125@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:48:20.790689 13157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.768505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39416 (local address 127.21.202.126:46071)
0806 19:48:20.769193 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:20.769203 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:20.770174 (+ 971us) server_negotiation.cc:407] Connection header received
0806 19:48:20.772136 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:20.772145 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:20.772248 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:20.772387 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:20.774133 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.775318 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.778079 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.778526 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.778643 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:20.783506 (+ 4863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:20.783513 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:20.783516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:20.783573 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:20.786710 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:20.787634 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:20.787640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:20.787643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:20.787819 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:20.788274 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:20.788279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:20.788282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:20.788623 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:20.788793 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:20.789996 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0806 19:48:20.790279 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":367,"threads_started":1}
I20260806 19:48:20.791817 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.769128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46071 (local address 127.0.0.1:39416)
0806 19:48:20.769972 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.770022 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:20.771829 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:20.772586 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:20.772595 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:20.773262 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:20.773865 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:20.773883 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.775515 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.775522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:20.777881 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:20.777901 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.778786 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.778795 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:20.779271 (+ 476us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:20.780158 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:20.780193 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:20.783186 (+ 2993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:20.786894 (+ 3708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:20.786906 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:20.786957 (+ 51us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:20.787485 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:20.787947 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:20.787952 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:20.787956 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:20.788141 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:20.788762 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:20.788770 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:20.789265 (+ 495us) client_negotiation.cc:769] Sending connection context
0806 19:48:20.789817 (+ 552us) client_negotiation.cc:240] Negotiation successful
0806 19:48:20.790447 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":361,"threads_started":1}
I20260806 19:48:20.794035 13155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.202.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.202.124@KRBTEST.COM" created.
Aug 06 19:48:20 dist-test-slave-77v9 krb5kdc[13001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045700, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.125@KRBTEST.COM for kudu/127.21.202.125@KRBTEST.COM
I20260806 19:48:20.811513 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.795107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33459 (local address 127.0.0.1:58062)
0806 19:48:20.795314 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.795339 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:20.795441 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:20.796660 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:20.796664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:20.797215 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:20.797462 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:20.797474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.799041 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.799045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:20.799908 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:20.799920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.801128 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.801133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:20.801295 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:20.801957 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:20.801977 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:20.804388 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:20.808933 (+ 4545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:20.808938 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:20.808941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:20.809312 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:20.809765 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:20.809768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:20.809771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:20.809935 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:20.810748 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:20.810752 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:20.810896 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 19:48:20.811013 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 19:48:20.811229 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 19:48:20.812008 13160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.795420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58062 (local address 127.21.202.125:33459)
0806 19:48:20.796194 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:20.796201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:20.796229 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:48:20.796279 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:20.796285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:20.796481 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:20.796672 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:20.797677 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.798895 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.800061 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.800949 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.801095 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:20.804635 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:20.804655 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:20.804677 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:20.804720 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:20.808716 (+ 3996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:20.809466 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:20.809478 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:20.809483 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:20.809641 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:20.810054 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:20.810062 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:20.810067 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:20.810595 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:20.810817 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:20.811202 (+ 385us) server_negotiation.cc:299] Negotiation successful
0806 19:48:20.811513 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":466,"threads_started":1}
I20260806 19:48:20.814581 13155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:48:20.815847 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.815335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38562)
0806 19:48:20.815531 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.815667 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260806 19:48:20.816092 13106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
I20260806 19:48:20.817771 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.817342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38568)
0806 19:48:20.817586 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.817667 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260806 19:48:20.817974 13155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:43819: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.202.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.124.keytab.
Entry for principal kudu/127.21.202.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.124.keytab.
I20260806 19:48:20.841511 22313 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskha9koQ/build/asan/bin/kudu
/tmp/dist-test-taskha9koQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.124:43819
--webserver_interface=127.21.202.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.124.keytab
--principal=kudu/127.21.202.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.202.84:41465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.202.126:46071,127.21.202.125:33459,127.21.202.124:43819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:48:20.863873 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } attempt: 5
I20260806 19:48:20.867269 13155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } attempt: 1
I20260806 19:48:20.868393 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.867876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38582)
0806 19:48:20.868053 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.868196 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 19:48:20.869725 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.869171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38586)
0806 19:48:20.869381 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.869538 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 19:48:20.869971 13155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:43819: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Aug 06 19:48:20 dist-test-slave-77v9 krb5kdc[13001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045700, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for kudu/127.21.202.125@KRBTEST.COM
I20260806 19:48:20.880991 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.864762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33459 (local address 127.0.0.1:58068)
0806 19:48:20.865053 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.865077 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:20.865205 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:20.865721 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:20.865724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:20.866325 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:20.866543 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:20.866553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.868170 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.868178 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:20.869471 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:20.869485 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.870514 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.870520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:20.870737 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:20.871426 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:20.871445 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:20.874125 (+ 2680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:20.877881 (+ 3756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:20.877887 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:20.877892 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:20.878461 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:20.879110 (+ 649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:20.879116 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:20.879120 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:20.879293 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:20.879934 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:20.879939 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:20.880115 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 19:48:20.880258 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 19:48:20.880587 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260806 19:48:20.881018 13160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.864905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58068 (local address 127.21.202.125:33459)
0806 19:48:20.865138 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:20.865146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:20.865177 (+ 31us) server_negotiation.cc:407] Connection header received
0806 19:48:20.865358 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:20.865367 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:20.865507 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:20.865687 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:20.866737 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.867955 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.869632 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:20.870350 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:20.870503 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:20.874352 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:20.874358 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:20.874362 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:20.874396 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:20.877722 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:20.878768 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:20.878778 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:20.878784 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:20.878984 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:20.879421 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:20.879425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:20.879428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:20.879802 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:20.879940 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:20.880373 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 19:48:20.880696 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:48:20.882684 13081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:48:20.883984 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.883497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38590)
0806 19:48:20.883718 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.883813 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 19:48:20.885051 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.884657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38600)
0806 19:48:20.884832 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.884933 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 19:48:20.885280 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:43819: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
I20260806 19:48:20.936573 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } attempt: 1
I20260806 19:48:20.938180 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.937503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38604)
0806 19:48:20.937786 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.937917 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 19:48:20.938230 13155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } attempt: 2
I20260806 19:48:20.939294 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.938897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38610)
0806 19:48:20.939064 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.939154 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 19:48:20.939453 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.938969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38612)
0806 19:48:20.939182 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.939269 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260806 19:48:20.939514 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:43819: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
I20260806 19:48:20.940528 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:20.940182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38632)
0806 19:48:20.940337 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:20.940414 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 19:48:20.940757 13155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:43819: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
I20260806 19:48:21.007783 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } attempt: 2
I20260806 19:48:21.009434 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.008903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38642)
0806 19:48:21.009174 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.009272 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260806 19:48:21.010478 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.010128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38648)
0806 19:48:21.010290 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.010366 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 19:48:21.010685 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:43819: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
I20260806 19:48:21.032043 13155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } attempt: 3
I20260806 19:48:21.033403 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.032771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38660)
0806 19:48:21.033051 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.033167 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260806 19:48:21.034546 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.034102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38672)
0806 19:48:21.034338 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.034430 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260806 19:48:21.034754 13155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:43819: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
I20260806 19:48:21.116964 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } attempt: 3
I20260806 19:48:21.118268 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.117644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38682)
0806 19:48:21.117917 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.118039 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 19:48:21.119473 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.118987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38694)
0806 19:48:21.119217 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.119311 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260806 19:48:21.119733 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:43819: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
I20260806 19:48:21.177989 13155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } attempt: 4
I20260806 19:48:21.179131 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.178632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38700)
0806 19:48:21.178877 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.178968 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 19:48:21.180224 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.179815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38710)
0806 19:48:21.179981 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.180084 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 19:48:21.180459 13155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:43819: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
W20260806 19:48:21.207923 13163 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 19:48:21.208248 13163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:48:21.208310 13163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:48:21.208392 13163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:48:21.215690 13163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:48:21.215803 13163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:48:21.215862 13163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:48:21.215905 13163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:48:21.215960 13163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:48:21.226107 13163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.202.84:41465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.202.126:46071,127.21.202.125:33459,127.21.202.124:43819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/krb5kdc/kudu_127.21.202.124.keytab
--principal=kudu/127.21.202.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.202.124:43819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.202.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:48:21.227551 13163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:48:21.228869 13163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:48:21.242341 13168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:21.242262 13171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:48:21.242259 13169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:48:21.243517 13163 server_base.cc:1034] running on GCE node
I20260806 19:48:21.244302 13163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:48:21.245815 13163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:48:21.247071 13163 hybrid_clock.cc:648] HybridClock initialized: now 1786045701247011 us; error 26 us; skew 500 ppm
I20260806 19:48:21.247416 13163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:48:21 dist-test-slave-77v9 krb5kdc[13001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:48:21.251058 13163 init.cc:377] Logged in from keytab as kudu/127.21.202.124@KRBTEST.COM (short username kudu)
I20260806 19:48:21.252792 13163 webserver.cc:460] Webserver started at http://127.21.202.124:40407/ using document root <none> and password file <none>
I20260806 19:48:21.253684 13163 fs_manager.cc:359] Metadata directory not provided
I20260806 19:48:21.253789 13163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:48:21.254184 13163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:48:21.257700 13163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/data/instance:
uuid: "06b1c4b2cc4d4fdfa9988621267d225d"
format_stamp: "Formatted at 2026-08-06 19:48:21 on dist-test-slave-77v9"
server_key: "fb5bc6307f008b3998c8e860db47653d"
server_key_iv: "fcff00e882d4e5ca5e2801398dad88c4"
server_key_version: "encryptionkey@0"
I20260806 19:48:21.258644 13163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/wal/instance:
uuid: "06b1c4b2cc4d4fdfa9988621267d225d"
format_stamp: "Formatted at 2026-08-06 19:48:21 on dist-test-slave-77v9"
server_key: "fb5bc6307f008b3998c8e860db47653d"
server_key_iv: "fcff00e882d4e5ca5e2801398dad88c4"
server_key_version: "encryptionkey@0"
I20260806 19:48:21.264554 13163 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 19:48:21.268698 13178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:48:21.270272 13163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:48:21.270478 13163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/data,/tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/wal
uuid: "06b1c4b2cc4d4fdfa9988621267d225d"
format_stamp: "Formatted at 2026-08-06 19:48:21 on dist-test-slave-77v9"
server_key: "fb5bc6307f008b3998c8e860db47653d"
server_key_iv: "fcff00e882d4e5ca5e2801398dad88c4"
server_key_version: "encryptionkey@0"
I20260806 19:48:21.270730 13163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:48:21.296973 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } attempt: 4
I20260806 19:48:21.298197 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.297731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38716)
0806 19:48:21.297948 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.298058 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 19:48:21.299281 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.298925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38724)
0806 19:48:21.299096 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.299165 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 19:48:21.299517 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.202.124:43819: Network error: Client connection negotiation failed: client connection to 127.21.202.124:43819: connect: Connection refused (error 111)
I20260806 19:48:21.304605 13163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:48:21.309547 13163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:48:21.309870 13163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:48:21.329838 13163 rpc_server.cc:307] RPC server started. Bound to: 127.21.202.124:43819
I20260806 19:48:21.329864 13229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.202.124:43819 every 8 connection(s)
I20260806 19:48:21.331254 13163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/data/info.pb
I20260806 19:48:21.335366 13230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:48:21.339211 22313 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskha9koQ/build/asan/bin/kudu as pid 13163
I20260806 19:48:21.339429 22313 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskha9koQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786045310306758-22313-0/minicluster-data/master-2/wal/instance
I20260806 19:48:21.339887 22313 external_mini_cluster.cc:1428] Setting key d171ec1a552aa113b2e2c24af16d4f17
I20260806 19:48:21.345001 13230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:48:21 dist-test-slave-77v9 krb5kdc[13001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.124@KRBTEST.COM
Aug 06 19:48:21 dist-test-slave-77v9 krb5kdc[13001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.124@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:48:21.368456 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.348285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39418 (local address 127.21.202.126:46071)
0806 19:48:21.349939 (+ 1654us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:21.349945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:21.349966 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:48:21.350026 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:21.350031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:21.350109 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:21.350239 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:21.351851 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.352979 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.355166 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.355607 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.355719 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.360459 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:21.360468 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:21.360473 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:21.360534 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:21.364300 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.365316 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.365323 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.365327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.365508 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.366116 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.366121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.366124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.366537 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:21.366654 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:21.367742 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0806 19:48:21.367981 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1531,"thread_start_us":1383,"threads_started":1}
I20260806 19:48:21.369767 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.341906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38738 (local address 127.21.202.124:43819)
0806 19:48:21.342811 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:21.342823 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:21.342863 (+ 40us) server_negotiation.cc:407] Connection header received
0806 19:48:21.344999 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:21.345023 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:21.345467 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:21.345864 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:21.347784 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.350119 (+ 2335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.352061 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.353128 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.353311 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.359954 (+ 6643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:21.359965 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:21.359985 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:21.360033 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:21.364652 (+ 4619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.365555 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.365562 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.365564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.365911 (+ 347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.366469 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.366474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.366476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.367233 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:21.367570 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:21.367927 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 19:48:21.368274 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":677,"spinlock_wait_cycles":30464,"thread_start_us":389,"threads_started":1}
I20260806 19:48:21.369827 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.348173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.126:46071 (local address 127.0.0.1:39418)
0806 19:48:21.349473 (+ 1300us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.349544 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:21.349729 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:21.350394 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:21.350404 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:21.351313 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:21.351648 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:21.351661 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.353242 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.353249 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:21.354961 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:21.354981 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.355885 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.355893 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:21.356086 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.356997 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:21.357146 (+ 149us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:21.360185 (+ 3039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:21.364498 (+ 4313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:21.364511 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:21.364522 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:21.365155 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:21.365647 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:21.365660 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:21.365665 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:21.365944 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:21.366692 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:21.366701 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:21.367244 (+ 543us) client_negotiation.cc:769] Sending connection context
0806 19:48:21.367577 (+ 333us) client_negotiation.cc:240] Negotiation successful
0806 19:48:21.368165 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":58,"spinlock_wait_cycles":1280,"thread_start_us":717,"threads_started":1}
I20260806 19:48:21.371739 13230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:48:21 dist-test-slave-77v9 krb5kdc[13001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.124@KRBTEST.COM for kudu/127.21.202.125@KRBTEST.COM
Aug 06 19:48:21 dist-test-slave-77v9 krb5kdc[13001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.126@KRBTEST.COM
I20260806 19:48:21.391932 13160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.373056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58084 (local address 127.21.202.125:33459)
0806 19:48:21.373291 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:21.373297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:21.373326 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:48:21.373720 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:21.373726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:21.373821 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:21.373967 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:21.375369 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.376269 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.378651 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.379400 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.379571 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.383798 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:21.383804 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:21.383807 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:21.383846 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:21.387645 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.389406 (+ 1761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.389411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.389414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.389574 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.390220 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.390225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.390228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.390639 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:21.390815 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:21.391429 (+ 614us) server_negotiation.cc:299] Negotiation successful
0806 19:48:21.391662 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 19:48:21.391922 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.372975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.125:33459 (local address 127.0.0.1:58084)
0806 19:48:21.373233 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.373261 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:21.373439 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:21.374225 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:21.374233 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:21.374837 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:21.375094 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:21.375113 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.376577 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.376582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:21.378380 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:21.378411 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.379647 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.379653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:21.379899 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.380654 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:21.380688 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:21.383515 (+ 2827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:21.387938 (+ 4423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:21.387946 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:21.387951 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:21.389087 (+ 1136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:21.389715 (+ 628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:21.389723 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:21.389729 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:21.390009 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:21.390864 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:21.390872 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:21.391086 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 19:48:21.391267 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 19:48:21.391556 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260806 19:48:21.393394 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.373586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39430 (local address 127.21.202.126:46071)
0806 19:48:21.373797 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:21.373802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:21.373823 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:48:21.374039 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:21.374044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:21.374135 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:21.374215 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:21.376051 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.376927 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.379040 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.379823 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.379980 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.385160 (+ 5180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:21.385166 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:21.385169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:21.385204 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:21.389888 (+ 4684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.390902 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.390908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.390912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.391108 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.391578 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.391588 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.391591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.391949 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:21.392376 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:21.392793 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 19:48:21.393079 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 19:48:21.393375 13230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:48:21 dist-test-slave-77v9 krb5kdc[13001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.124@KRBTEST.COM for kudu/127.21.202.124@KRBTEST.COM
Aug 06 19:48:21 dist-test-slave-77v9 krb5kdc[13001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.202.125@KRBTEST.COM
I20260806 19:48:21.409415 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.394354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38752)
0806 19:48:21.394713 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.394733 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:21.394860 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:21.395257 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:21.395265 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:21.395857 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:21.396007 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:21.396017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.397144 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.397150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:21.398276 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:21.398288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.399193 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.399201 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:21.399482 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.400202 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:21.400229 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:21.403220 (+ 2991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:21.406563 (+ 3343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:21.406567 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:21.406570 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:21.406926 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:21.407354 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:21.407361 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:21.407365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:21.407525 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:21.408252 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:21.408259 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:21.408463 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 19:48:21.408643 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 19:48:21.409003 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258}
I20260806 19:48:21.409638 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.394369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38752 (local address 127.21.202.124:43819)
0806 19:48:21.394583 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:21.394593 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:21.394849 (+ 256us) server_negotiation.cc:407] Connection header received
0806 19:48:21.394969 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:21.394975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:21.395057 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:21.395181 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:21.396165 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.397005 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.398440 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.399009 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.399104 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.403466 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:21.403471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:21.403474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:21.403504 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:21.406413 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.407052 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.407060 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.407066 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.407188 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.407650 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.407657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.407661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.408063 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:21.408228 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:21.408798 (+ 570us) server_negotiation.cc:299] Negotiation successful
0806 19:48:21.409218 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":63}
I20260806 19:48:21.413300 13160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.395695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58096 (local address 127.21.202.125:33459)
0806 19:48:21.395871 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:21.395876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:21.395905 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:48:21.396058 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:21.396063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:21.396165 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:21.396265 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:21.397773 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.398678 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.400589 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.401136 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.401237 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.406525 (+ 5288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:21.406533 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:21.406538 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:21.406584 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:21.410383 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.411209 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.411213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.411216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.411353 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.411809 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.411812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.411814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.412079 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:21.412289 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:21.412697 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 19:48:21.412969 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 19:48:21.416167 13230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d: Bootstrap starting.
I20260806 19:48:21.418776 22313 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:48:21.419772 13230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d: Neither blocks nor log segments found. Creating new log.
I20260806 19:48:21.420945 13230 log.cc:824] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d: Log is configured to *not* fsync() on all Append() calls
I20260806 19:48:21.423942 13230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d: No bootstrap required, opened a new log
I20260806 19:48:21.421758 13056 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a1ac208b80a64adaae648ace12a63f35"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39430
I20260806 19:48:21.434717 13230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } }
I20260806 19:48:21.435218 13230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b1c4b2cc4d4fdfa9988621267d225d, State: Initialized, Role: FOLLOWER
I20260806 19:48:21.435668 13230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } }
I20260806 19:48:21.436782 13237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } } }
I20260806 19:48:21.437194 13237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:48:21.438117 13230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d [sys.catalog]: configured and running, proceeding with master startup.
W20260806 19:48:21.449465 13248 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:48:21.449618 13248 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:48:21.446165 13163 master_runner.cc:186] Error getting master registration for 127.21.202.126:46071: OK, instance_id { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" instance_seqno: 1786045700142801 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 19:48:21.458336 13163 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 19:48:21.479698 13155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } attempt: 5
Aug 06 19:48:21 dist-test-slave-77v9 krb5kdc[13001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045700, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.125@KRBTEST.COM for kudu/127.21.202.124@KRBTEST.COM
I20260806 19:48:21.497287 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.480646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38766)
0806 19:48:21.480886 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.480910 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:21.481067 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:21.481599 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:21.481604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:21.482237 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:21.482654 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:21.482667 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.483886 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.483893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:21.484803 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:21.484879 (+ 76us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.485702 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.485708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:21.486036 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.486891 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:21.486923 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:21.489943 (+ 3020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:21.494203 (+ 4260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:21.494227 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:21.494234 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:21.494790 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:21.495218 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:21.495222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:21.495226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:21.495423 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:21.496210 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:21.496214 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:21.496375 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 19:48:21.496515 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 19:48:21.496875 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 19:48:21.497371 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.480710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.21.202.124:43819)
0806 19:48:21.481014 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:21.481020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:21.481042 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:48:21.481374 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:21.481380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:21.481447 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:21.481564 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:21.482844 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.483708 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.485025 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.485527 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.485669 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.490157 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:21.490164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:21.490169 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:21.490213 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:21.494009 (+ 3796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.494946 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.494952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.494955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.495095 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.495564 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.495569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.495572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.496084 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:21.496236 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:21.496708 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 19:48:21.496991 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260806 19:48:21.502010 13155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739: Bootstrap starting.
I20260806 19:48:21.507234 13155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739: Neither blocks nor log segments found. Creating new log.
I20260806 19:48:21.508381 13155 log.cc:824] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739: Log is configured to *not* fsync() on all Append() calls
I20260806 19:48:21.511343 13155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739: No bootstrap required, opened a new log
I20260806 19:48:21.519971 13155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } }
I20260806 19:48:21.520653 13155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 287aefad5b6043eebf365ff4d2ad8739, State: Initialized, Role: FOLLOWER
I20260806 19:48:21.521528 13155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } }
I20260806 19:48:21.522836 13251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } } }
I20260806 19:48:21.523250 13251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:48:21.524242 13155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 19:48:21.535024 13262 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:48:21.535176 13262 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:48:21.532249 13088 master_runner.cc:186] Error getting master registration for 127.21.202.126:46071: OK, instance_id { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" instance_seqno: 1786045700142801 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 19:48:21.541494 13088 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 19:48:21.562106 13184 tablet.cc:2369] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:48:21.576799 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } attempt: 5
Aug 06 19:48:21 dist-test-slave-77v9 krb5kdc[13001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045700, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.202.126@KRBTEST.COM for kudu/127.21.202.124@KRBTEST.COM
I20260806 19:48:21.595266 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.577676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.202.124:43819 (local address 127.0.0.1:38778)
0806 19:48:21.577891 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 19:48:21.577917 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:48:21.578054 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:48:21.578910 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:48:21.578917 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:48:21.579416 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:48:21.579595 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:21.579605 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.581542 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.581549 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:21.582522 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:48:21.582535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.583591 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.583597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:48:21.583830 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.584535 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:48:21.584554 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:48:21.587254 (+ 2700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:48:21.591941 (+ 4687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:21.591948 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:21.591953 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:21.592391 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:21.593135 (+ 744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:48:21.593141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:48:21.593146 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:48:21.593373 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:48:21.594079 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:48:21.594085 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:48:21.594274 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 19:48:21.594441 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 19:48:21.594777 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 19:48:21.595259 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.578264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38778 (local address 127.21.202.124:43819)
0806 19:48:21.578477 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:21.578484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:21.578505 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:48:21.578579 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:21.578587 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:21.578747 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:21.578858 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:21.579808 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.581394 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.582694 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.583437 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.583579 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.587542 (+ 3963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:21.587550 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:21.587555 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:21.587604 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:21.591770 (+ 4166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.592547 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.592555 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.592558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.592739 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.593527 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.593533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.593537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.593939 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:21.594096 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:21.594580 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 19:48:21.594831 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:48:21.600914 13081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35: Bootstrap starting.
I20260806 19:48:21.605208 13081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35: Neither blocks nor log segments found. Creating new log.
I20260806 19:48:21.606612 13081 log.cc:824] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35: Log is configured to *not* fsync() on all Append() calls
I20260806 19:48:21.610337 13081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35: No bootstrap required, opened a new log
I20260806 19:48:21.618211 13081 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } }
I20260806 19:48:21.618644 13081 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1ac208b80a64adaae648ace12a63f35, State: Initialized, Role: FOLLOWER
I20260806 19:48:21.619184 13081 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } }
I20260806 19:48:21.620497 13265 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } } }
I20260806 19:48:21.621234 13265 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:48:21.621809 13081 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:48:21.621860 13056 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 19:48:21.622083 13056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:48:21.627060 13056 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } }
W20260806 19:48:21.630402 13035 tablet.cc:2369] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:48:21.631902 13130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a1ac208b80a64adaae648ace12a63f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "287aefad5b6043eebf365ff4d2ad8739"
I20260806 19:48:21.632568 13130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:48:21.637856 13130 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1ac208b80a64adaae648ace12a63f35 in term 1.
I20260806 19:48:21.638974 13030 leader_election.cc:302] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 287aefad5b6043eebf365ff4d2ad8739, a1ac208b80a64adaae648ace12a63f35; no voters:
I20260806 19:48:21.641906 13205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a1ac208b80a64adaae648ace12a63f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "06b1c4b2cc4d4fdfa9988621267d225d"
I20260806 19:48:21.642935 13205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:48:21.648981 13056 leader_election.cc:288] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [CANDIDATE]: Term 1 election: Requested vote from peers 287aefad5b6043eebf365ff4d2ad8739 (127.21.202.125:33459), 06b1c4b2cc4d4fdfa9988621267d225d (127.21.202.124:43819)
I20260806 19:48:21.655349 13205 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1ac208b80a64adaae648ace12a63f35 in term 1.
I20260806 19:48:21.658282 13265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:48:21.666896 13265 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [term 1 LEADER]: Becoming Leader. State: Replica: a1ac208b80a64adaae648ace12a63f35, State: Running, Role: LEADER
I20260806 19:48:21.667855 13265 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } }
I20260806 19:48:21.702808 13271 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1ac208b80a64adaae648ace12a63f35. Latest consensus state: current_term: 1 leader_uuid: "a1ac208b80a64adaae648ace12a63f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } } }
I20260806 19:48:21.703804 13271 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:21.705945 13279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260806 19:48:21.731755 13109 tablet.cc:2369] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:48:21.760685 13279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:48:21.776425 13287 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 19:48:21.785830 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.676100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39436 (local address 127.21.202.126:46071)
0806 19:48:21.676290 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:21.676297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:21.693608 (+ 17311us) server_negotiation.cc:407] Connection header received
0806 19:48:21.693914 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:21.693924 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:21.694039 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:21.694132 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:21.709065 (+ 14933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.710665 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.722031 (+ 11366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.723244 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.723421 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.754340 (+ 30919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:21.754357 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:21.754364 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:21.754512 (+ 148us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:21.758157 (+ 3645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.773035 (+ 14878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.773047 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.773055 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.773357 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.781512 (+ 8155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.781518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.781521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.782008 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:21.783192 (+ 1184us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:21.785124 (+ 1932us) server_negotiation.cc:299] Negotiation successful
0806 19:48:21.785401 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 19:48:21.787721 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.677655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38782 (local address 127.21.202.124:43819)
0806 19:48:21.677919 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:21.677929 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:21.697829 (+ 19900us) server_negotiation.cc:407] Connection header received
0806 19:48:21.698182 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:21.698192 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:21.698339 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:21.698444 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:21.713160 (+ 14716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.714712 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.726342 (+ 11630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.727598 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.727790 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.748730 (+ 20940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:21.748739 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:21.748747 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:21.748804 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:21.753214 (+ 4410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.779844 (+ 26630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.779853 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.779870 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.780074 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.783758 (+ 3684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.783763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.783767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.784221 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:21.784491 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:21.786740 (+ 2249us) server_negotiation.cc:299] Negotiation successful
0806 19:48:21.787025 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 19:48:21.793646 13160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:21.674718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58102 (local address 127.21.202.125:33459)
0806 19:48:21.674916 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:21.674923 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:21.688043 (+ 13120us) server_negotiation.cc:407] Connection header received
0806 19:48:21.688681 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:21.688690 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:21.688845 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:21.688945 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:21.717642 (+ 28697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.719072 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.737947 (+ 18875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:21.739090 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:21.739290 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:21.774268 (+ 34978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:48:21.774279 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:48:21.774287 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:48:21.774353 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:48:21.779582 (+ 5229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.788521 (+ 8939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.788529 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.788532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.788731 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:48:21.790659 (+ 1928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:48:21.790665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:48:21.790669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:48:21.791180 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:48:21.791578 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:21.792903 (+ 1325us) server_negotiation.cc:299] Negotiation successful
0806 19:48:21.793205 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 19:48:21.805656 13205 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d [term 1 FOLLOWER]: Refusing update from remote peer a1ac208b80a64adaae648ace12a63f35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:48:21.806111 13130 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739 [term 1 FOLLOWER]: Refusing update from remote peer a1ac208b80a64adaae648ace12a63f35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:48:21.807106 13265 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:48:21.807789 13265 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:48:21.837950 13237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1ac208b80a64adaae648ace12a63f35. Latest consensus state: current_term: 1 leader_uuid: "a1ac208b80a64adaae648ace12a63f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } } }
I20260806 19:48:21.838414 13237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:48:21.841282 13251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1ac208b80a64adaae648ace12a63f35. Latest consensus state: current_term: 1 leader_uuid: "a1ac208b80a64adaae648ace12a63f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } } }
I20260806 19:48:21.841646 13251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:48:21.850862 13289 mvcc.cc:205] Tried to move back new op lower bound from 7315643194567188480 to 7315643194088218624. Current Snapshot: MvccSnapshot[applied={T|T < 7315643194567188480}]
I20260806 19:48:21.854972 13290 mvcc.cc:205] Tried to move back new op lower bound from 7315643194567188480 to 7315643194088218624. Current Snapshot: MvccSnapshot[applied={T|T < 7315643194567188480}]
I20260806 19:48:21.859987 13251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a1ac208b80a64adaae648ace12a63f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } } }
I20260806 19:48:21.860337 13251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:48:21.866834 13265 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a1ac208b80a64adaae648ace12a63f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } } }
I20260806 19:48:21.867225 13265 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:21.867473 13279 catalog_manager.cc:1347] Generated new cluster ID: ed166fb7d6d74ef39592f3530ffc37a2
I20260806 19:48:21.867556 13279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:48:21.867964 13265 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a1ac208b80a64adaae648ace12a63f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } } }
I20260806 19:48:21.868211 13265 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35 [sys.catalog]: This master's current role is: LEADER
I20260806 19:48:21.873211 13237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a1ac208b80a64adaae648ace12a63f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1ac208b80a64adaae648ace12a63f35" member_type: VOTER last_known_addr { host: "127.21.202.126" port: 46071 } } peers { permanent_uuid: "287aefad5b6043eebf365ff4d2ad8739" member_type: VOTER last_known_addr { host: "127.21.202.125" port: 33459 } } peers { permanent_uuid: "06b1c4b2cc4d4fdfa9988621267d225d" member_type: VOTER last_known_addr { host: "127.21.202.124" port: 43819 } } }
I20260806 19:48:21.873467 13237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:48:21.942721 13279 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:48:21.945853 13279 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:48:21.980414 13279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1ac208b80a64adaae648ace12a63f35: Generated new TSK 0
I20260806 19:48:21.981906 13279 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 19:48:22.115037 22313 mini_kdc.cc:348] Time spent kdestroy: real 0.111s user 0.001s sys 0.030s
W20260806 19:48:22.184496 13235 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:22.162347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39438 (local address 127.21.202.126:46071)
0806 19:48:22.181501 (+ 19154us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.181510 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.181695 (+ 185us) server_negotiation.cc:407] Connection header received
0806 19:48:22.182728 (+ 1033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.182735 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.182826 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.182934 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:22.184204 (+ 1270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39438: BlockingRecv error: recv got EOF from 127.0.0.1:39438 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260806 19:48:22.187175 13160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:22.169111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58110 (local address 127.21.202.125:33459)
0806 19:48:22.169384 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.169391 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.182042 (+ 12651us) server_negotiation.cc:407] Connection header received
0806 19:48:22.182274 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.182281 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.182391 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.182495 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:22.186903 (+ 4408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58110: BlockingRecv error: recv got EOF from 127.0.0.1:58110 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 19:48:22.198233 13233 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:22.179340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38786 (local address 127.21.202.124:43819)
0806 19:48:22.179597 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.179605 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.185421 (+ 5816us) server_negotiation.cc:407] Connection header received
0806 19:48:22.185577 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.185583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.185671 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.185776 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:22.197964 (+ 12188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38786: BlockingRecv error: recv got EOF from 127.0.0.1:38786 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260806 19:48:22.233809 13160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:22.230804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58124 (local address 127.21.202.125:33459)
0806 19:48:22.231018 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.231027 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.231597 (+ 570us) server_negotiation.cc:407] Connection header received
0806 19:48:22.231953 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.231962 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.232121 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.232215 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:22.233478 (+ 1263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58124: BlockingRecv error: recv got EOF from 127.0.0.1:58124 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 19:48:22.243209 13160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:22.240172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58138 (local address 127.21.202.125:33459)
0806 19:48:22.240397 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.240405 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.241466 (+ 1061us) server_negotiation.cc:407] Connection header received
0806 19:48:22.241693 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.241699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.241814 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.241938 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:22.242971 (+ 1033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58138: BlockingRecv error: recv got EOF from 127.0.0.1:58138 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 19:48:22.255239 13160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:22.249408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.21.202.125:33459)
0806 19:48:22.249621 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.249629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.253517 (+ 3888us) server_negotiation.cc:407] Connection header received
0806 19:48:22.253754 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.253760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.253870 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.253998 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:22.254978 (+ 980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58152: BlockingRecv error: recv got EOF from 127.0.0.1:58152 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 19:48:22.265815 13160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:22.263174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.21.202.125:33459)
0806 19:48:22.263438 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.263447 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.264074 (+ 627us) server_negotiation.cc:407] Connection header received
0806 19:48:22.264328 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.264334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.264440 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.264554 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:22.265547 (+ 993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58166: BlockingRecv error: recv got EOF from 127.0.0.1:58166 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
W20260806 19:48:22.281778 13160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:22.279046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.21.202.125:33459)
0806 19:48:22.279281 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.279290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.279910 (+ 620us) server_negotiation.cc:407] Connection header received
0806 19:48:22.280142 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.280149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.280260 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.280399 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:22.281506 (+ 1107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58168: BlockingRecv error: recv got EOF from 127.0.0.1:58168 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 19:48:22.305260 13160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:22.302449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58182 (local address 127.21.202.125:33459)
0806 19:48:22.302695 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.302703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.303367 (+ 664us) server_negotiation.cc:407] Connection header received
0806 19:48:22.303636 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.303645 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.303769 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.303908 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:22.304938 (+ 1030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58182: BlockingRecv error: recv got EOF from 127.0.0.1:58182 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 19:48:22.344372 13160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:22.341751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58192 (local address 127.21.202.125:33459)
0806 19:48:22.341953 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.341960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.342562 (+ 602us) server_negotiation.cc:407] Connection header received
0806 19:48:22.342797 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.342806 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.342928 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.343051 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:22.344067 (+ 1016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58192: BlockingRecv error: recv got EOF from 127.0.0.1:58192 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 19:48:22.415561 13160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:48:22.412846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58194 (local address 127.21.202.125:33459)
0806 19:48:22.413067 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.413074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.413719 (+ 645us) server_negotiation.cc:407] Connection header received
0806 19:48:22.413939 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.413956 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.414097 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.414238 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:48:22.415256 (+ 1018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58194: BlockingRecv error: recv got EOF from 127.0.0.1:58194 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 19:48:22.451792 13248 catalog_manager.cc:1259] Loaded cluster ID: ed166fb7d6d74ef39592f3530ffc37a2
I20260806 19:48:22.452042 13248 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d: loading cluster ID for follower catalog manager: success
I20260806 19:48:22.455071 13248 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d: acquiring CA information for follower catalog manager: success
I20260806 19:48:22.457568 13248 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 06b1c4b2cc4d4fdfa9988621267d225d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:48:22.537518 13262 catalog_manager.cc:1259] Loaded cluster ID: ed166fb7d6d74ef39592f3530ffc37a2
I20260806 19:48:22.537674 13262 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739: loading cluster ID for follower catalog manager: success
I20260806 19:48:22.540553 13262 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739: acquiring CA information for follower catalog manager: success
I20260806 19:48:22.542598 13262 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 287aefad5b6043eebf365ff4d2ad8739: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:48:22.553815 13160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:22.547279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58208 (local address 127.21.202.125:33459)
0806 19:48:22.547536 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.547544 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.547916 (+ 372us) server_negotiation.cc:407] Connection header received
0806 19:48:22.548155 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.548161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.548313 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.548460 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:48:22.549003 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:22.549725 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:22.551301 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:22.551781 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:22.551904 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:22.552439 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:48:22.553026 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:48:22.553184 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:22.553351 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 19:48:22.553439 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 19:48:22.566655 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:22.560112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38800 (local address 127.21.202.124:43819)
0806 19:48:22.560375 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.560393 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.560680 (+ 287us) server_negotiation.cc:407] Connection header received
0806 19:48:22.561058 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.561063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.561231 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.561427 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:48:22.561866 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:22.562681 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:22.564175 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:22.564619 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:22.564778 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:22.565391 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:48:22.565948 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:48:22.566128 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:22.566205 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 19:48:22.566306 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 19:48:22.581374 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:22.573142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39444 (local address 127.21.202.126:46071)
0806 19:48:22.573361 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.573369 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.574013 (+ 644us) server_negotiation.cc:407] Connection header received
0806 19:48:22.574479 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.574485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.574733 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.574897 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:48:22.575723 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:22.576837 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:22.578874 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:22.579292 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:22.579440 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:22.580053 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:48:22.580536 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:48:22.580651 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:22.580906 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 19:48:22.581018 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 19:48:22.582441 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:22.573726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38814 (local address 127.21.202.124:43819)
0806 19:48:22.573903 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.573910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.575250 (+ 1340us) server_negotiation.cc:407] Connection header received
0806 19:48:22.575569 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.575577 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.575669 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.575733 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:48:22.576570 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:22.577880 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:22.580278 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:22.580889 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:22.581020 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:22.581484 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:48:22.581611 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:48:22.581693 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:22.582064 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 19:48:22.582146 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 19:48:22.583223 13160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:48:22.573977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58210 (local address 127.21.202.125:33459)
0806 19:48:22.574234 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 19:48:22.574241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:48:22.575060 (+ 819us) server_negotiation.cc:407] Connection header received
0806 19:48:22.575333 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:48:22.575342 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:48:22.575502 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:48:22.575608 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:48:22.576232 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:22.577455 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:22.580040 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:48:22.580663 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:48:22.580852 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:48:22.581412 (+ 560us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:48:22.581570 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:48:22.581711 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 19:48:22.582049 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 19:48:22.582138 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 19:48:22.587956 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 13014
I20260806 19:48:22.589069 13076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:48:22.702011 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 13014
I20260806 19:48:22.726411 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 13088
I20260806 19:48:22.727711 13150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:48:22.840049 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 13088
I20260806 19:48:22.863858 22313 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskha9koQ/build/asan/bin/kudu pid 13163
I20260806 19:48:22.865248 13225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:48:22.976857 22313 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskha9koQ/build/asan/bin/kudu with pid 13163
2026-08-06T19:48:22Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4893 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9801 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (392599 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST